Image processing apparatus and method, program recording medium, and program

ABSTRACT

An image processing apparatus includes the following elements. A broad-range feature extraction unit extracts broad-range features from pixels located in a predetermined area in relation to a subject pixel of a first image. A broad-range degree-of-artificiality calculator calculates, in a multidimensional space represented by the broad-range features, the broad-range degree of artificiality from the positional relationship of the broad-range features to a statistical distribution range of an artificial image of the first image. A narrow-range feature extraction unit extracts narrow-range features from pixels located in the predetermined area in relation to the subject pixel of the first image. A narrow-range degree-of-artificiality calculator calculates, in a multidimensional space represented by the narrow-range features, the narrow-range degree of artificiality from the positional relationship of the narrow-range features to a statistical distribution range of the artificial image. A degree-of-artificiality calculator calculates the degree of artificiality of the subject pixel.

CROSS REFERENCES TO RELATED APPLICATIONS

The present invention contains subject matter related to Japanese PatentApplication JP 2006-073557 filed in the Japanese Patent Office on Mar.16, 2006, the entire contents of which are incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention generally relates to image processing apparatusesand methods, program recording media, and programs. More particularly,the invention relates to an image processing apparatus and method, aprogram recording medium, and a program that allow the quality of animage to be accurately enhanced by distinguishing artificial imagecomponents and natural image components included in an image from eachother and by performing optimal processing on each of the artificialimage components and natural image components.

2. Description of the Related Art

The assignee of this application previously proposed classificationadaptation processing in, for example, Japanese Unexamined PatentApplication Publication No. 9- 167240. In the classification adaptationprocessing, from an input first image, a second image is determined.More specifically, according to the pixel values of a plurality ofpixels in a predetermined area of the input first image, subject pixelsof the second image are allocated into classes, and then, a linearexpression of prediction coefficients which have been determined for theindividual classes by learning processing, and the pixel values of theplurality of pixels in the predetermined area of the input first imageare calculated, so that the second image can be determined from theinput first image.

For example, if the first image is an image containing noise and thesecond image is an image with suppressed noise, the classificationadaptation processing serves as noise suppression processing. If thefirst image is a standard definition (SD) image and the second image isa high definition (HD) image with a level of resolution higher than thatof the SD image, the classification adaptation processing serves asresolution conversion processing for converting a low-resolution imageinto a high-resolution image.

In known learning processing, to enable prediction of general movingpictures typified by broadcasting moving pictures, instead of artificialimages, which are discussed in detail below, natural images obtained bydirectly imaging subjects in nature are used as supervisor images andlearner images. Accordingly, if the first image is a natural image, thesecond image, which is a high-definition, fine image, can be predictedby performing classification adaptation processing using predictioncoefficients obtained by learning processing.

SUMMARY OF THE INVENTION

Normally, the first image is formed of a mixture of an area ofartificial image components and an area of natural image components.Accordingly, it is possible to predict the second image, which is ahigh-definition, fine image, by performing classification adaptationprocessing on the area of the natural image components by usingprediction coefficients determined by the above-described learningprocessing.

If, however, classification adaptation processing is performed on thearea of the artificial image components, such as text or simplegraphics, of the first image, by using the prediction coefficientsdetermined by the learning processing using natural images, the finenessof edged portions is excessively enhanced, and flat noise isdisadvantageously recognized as a correct waveform and the noise isfurther enhanced. As a result, ringing and noise enhancement occur inpart of the image.

Artificial images are images, for example, text or simple graphics,exhibiting a small number of grayscale levels and distinct phaseinformation concerning the positions of edges (outlines), i.e.,including many flat portions.

It is thus desirable to enhance the quality of a whole image bydistinguishing an area of natural image components and an area ofartificial image components contained in the image from each other andby performing optimal processing on each of the areas.

According to an embodiment of the present invention, there is providedan image processing apparatus including broad-range feature extractionmeans for extracting broad-range features including a plurality of typesof broad-range features from pixels located in a predetermined area inrelation to a subject pixel of a first image, broad-rangedegree-of-artificiality calculation means for calculating, in amultidimensional space represented by the plurality of types ofbroad-range features included in the broad-range features extracted bythe broad-range feature extraction means, from a positional relationshipof the plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image, narrow-range featureextraction means for extracting narrow-range features including aplurality of types of narrow-range features from pixels located in thepredetermined area in relation to the subject pixel of the first image,narrow-range degree-of-artificiality calculation means for calculating,in a multidimensional space represented by the plurality of types ofnarrow-range features included in the narrow-range features extracted bythe narrow-range feature extraction means, from a positionalrelationship of the plurality of types of narrow-range featuresaccording to a statistical distribution range of the artificial image ofthe first image, a narrow-range degree of artificiality representing adegree by which the plurality of types of narrow-range features belongto the statistical distribution range of the artificial image, anddegree-of-artificiality calculation means for calculating a degree ofartificiality of the subject pixel by combining the broad-range degreeof artificiality and the narrow-range degree of artificiality.

The image processing apparatus may further include first predictionmeans for predicting, from the first image, a second image which isobtained by increasing the quality of the artificial image, secondprediction means for predicting, from the first image, a third imagewhich is obtained by increasing the quality of a natural imageexhibiting a large number of grayscale levels and indistinct edges, andsynthesizing means for combining the second image and the third image onthe basis of the degree of artificiality.

The first prediction means may include first classification means forclassifying pixels of the second image into first classes, first storagemeans for storing a first prediction coefficient for each of the firstclasses, the first prediction coefficient being obtained by conductinglearning by using a plurality of artificial images, and firstcomputation means for determining, from the first image, the secondimage having a higher quality than the first image by performingcomputation using the first image and the first prediction coefficientsfor the first classes into which the pixels of the second image areclassified. The second prediction means may include secondclassification means for classifying pixels of the third image intosecond classes, second storage means for storing a second predictioncoefficient for each of the second classes, the second predictioncoefficient being obtained by conducting learning by using a pluralityof natural images, and second computation means for determining thethird image from the first image by performing computation using thefirst image and the second prediction coefficients for the secondclasses into which the pixels of the third image are classified.

The broad-range degree-of-artificiality calculation means may includebroad-range artificial-image distribution range storage means forstoring the statistical distribution range of the artificial image ofthe first image in the multidimensional space represented by theplurality of types of broad-range features, and the broad-rangedegree-of-artificiality calculation means may calculate the broad-rangedegree of artificiality from the positional relationship of theplurality of types of broad-range features extracted by the broad-rangefeature extraction means according to the statistical distribution rangeof the artificial image in the multidimensional space stored in thebroad-range artificial-image distribution range storage means.

The narrow-range degree-of-artificiality calculation means may includenarrow-range artificial-image distribution range storage means forstoring the statistical distribution range of the artificial image ofthe first image in the multidimensional space represented by theplurality of types of narrow-range features, and the narrow-rangedegree-of-artificiality calculation means may calculate the narrow-rangedegree of artificiality from the positional relationship of theplurality of types of narrow-range features extracted by thenarrow-range feature extraction means according to the statisticaldistribution range of the artificial image in the multidimensional spacestored in the narrow-range artificial-image distribution range storagemeans.

The broad-range feature extraction means may include edge featureextraction means for extracting a feature representing the presence ofan edge from the pixels located in the predetermined area, and flatfeature extraction means for extracting a feature representing thepresence of a flat portion from the pixels located in the predeterminedarea.

The edge feature extraction means may extract, as the featurerepresenting the presence of an edge, a difference dynamic range of thepixels located in the predetermined area by using a difference of pixelvalues between the subject pixel and each of the pixels located in thepredetermined area.

The edge feature extraction means may extract, as the featurerepresenting the presence of an edge, a difference dynamic range of thepixels located in the predetermined area by using a difference of thepixel values between the subject pixel and each of the pixels located inthe predetermined area after applying a weight to the difference of thepixel values in accordance with a distance therebetween.

The edge feature extraction means may extract, as the featurerepresenting the presence of an edge, a predetermined order of higherlevels of difference absolute values between adjacent pixels of thepixels located in the predetermined area.

The edge feature extraction means may extract, as the featurerepresenting the presence of an edge, the average of higher firstthrough second levels of difference absolute values between adjacentpixels of the pixels located in the predetermined area or the sum of thehigher first through second levels of the difference absolute valuesafter applying a weight to each of the difference absolute values.

The flat feature extraction means may extract, as the featurerepresenting the presence of a flat portion, the number of differenceabsolute values between adjacent pixels of the pixels located in thepredetermined area which are smaller than a predetermined threshold.

The predetermined threshold may be set based on the feature representingthe presence of an edge.

The flat feature extraction means may extract, as the featurerepresenting the presence of a flat portion, the sum of differenceabsolute values between adjacent pixels of the pixels located in thepredetermined area after transforming the difference absolute values bya predetermined function.

The flat feature extraction means may extract, as the featurerepresenting the presence of a flat portion, the sum of differenceabsolute values between the adjacent pixels of the pixels located in thepredetermined area after transforming the difference absolute values bya predetermined function and after applying a weight to each of thetransformed difference absolute values in accordance with a distancefrom the subject pixel to each of the pixels located in thepredetermined area.

The predetermined function may be a function associated with the featurerepresenting the presence of an edge.

The flat feature extraction means may extract, as the featurerepresenting the presence of a flat portion, a predetermined order oflower levels of difference absolute values between adjacent pixels ofthe pixels located in the predetermined area.

The flat feature extraction means may extract, as the featurerepresenting the presence of a flat portion, the average of lower firstthrough second levels of difference absolute values between adjacentpixels of the pixels located in the predetermined area or the sum of thelower first through second levels of the difference absolute valuesafter applying a weight to each of the difference absolute values.

The narrow-range feature extraction means may extract, from the pixelslocated in the predetermined area, the narrow-range features includingtwo types of features selected from features representing thin lines,edges, points, flat portions in the vicinity of edges, and gradation.

The narrow-range feature extraction means may include first narrow-rangefeature extraction means for extracting, as a first feature of thenarrow-range features, a pixel-value dynamic range obtained bysubtracting a minimum pixel value from a maximum pixel value of pixelslocated in a first area included in the predetermined area, and secondnarrow-range feature extraction means for extracting, as a secondfeature of the narrow-range features, a pixel-value dynamic rangeobtained by subtracting a minimum pixel value from a maximum pixel valueof pixels located in a second area including the subject pixel andincluded in the first area.

The second narrow-range feature extraction means may extract, as thesecond feature, a minimum pixel-value dynamic range of pixel-valuedynamic ranges obtained from a plurality of the second areas.

The first narrow-range feature extraction means may extract, as thefirst feature of the narrow-range features, a pixel-value dynamic rangeof the pixel values of the pixels located in the predetermined area. Thesecond narrow-range feature extraction means may extract, as the secondfeature of the narrow-range features, the sum of difference absolutevalues between the subject pixel and the pixels located in thepredetermined area after transforming the difference absolute values bya predetermined function and after applying a weight to each of thetransformed difference absolute values.

The second narrow-range feature extraction means may include weightcalculation means for calculating the weight in accordance with the sumof the difference absolute values between adjacent pixels of all thepixels located on a path from the subject pixel to the pixels located inthe first area.

The predetermined function may be a function associated with the firstfeature.

According to another embodiment of the present invention, there isprovided an image processing method including the steps of extractingbroad-range features including a plurality of types of broad-rangefeatures from pixels located in a predetermined area in relation to asubject pixel of a first image, calculating, in a multidimensional spacerepresented by the plurality of types of broad-range features includedin the extracted broad-range features, from a positional relationship ofthe plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image, extracting narrow-rangefeatures including a plurality of types of narrow-range features frompixels located in the predetermined area in relation to the subjectpixel of the first image, calculating, in a multidimensional spacerepresented by the plurality of types of narrow-range features includedin the extracted narrow-range features, from a positional relationshipof the plurality of types of narrow-range features according to astatistical distribution range of the artificial image of the firstimage, a narrow-range degree of artificiality representing a degree bywhich the plurality of types of narrow-range features belong to thestatistical distribution range of the artificial image, and calculatinga degree of artificiality of the subject pixel by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality.

According to another embodiment of the present invention, there isprovided a program recording medium recorded thereon a computer-readableprogram. The computer-readable program includes the steps of extractingbroad-range features including a plurality of types of broad-rangefeatures from pixels located in a predetermined area in relation to asubject pixel of a first image, calculating, in a multidimensional spacerepresented by the plurality of types of broad-range features includedin the extracted broad-range features, from a positional relationship ofthe plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image, extracting narrow-rangefeatures including a plurality of types of narrow-range features frompixels located in the predetermined area in relation to the subjectpixel of the first image, calculating, in a multidimensional spacerepresented by the plurality of types of narrow-range features includedin the extracted narrow-range features, from a positional relationshipof the plurality of types of narrow-range features according to astatistical distribution range of the artificial image of the firstimage, a narrow-range degree of artificiality representing a degree bywhich the plurality of types of narrow-range features belong to thestatistical distribution range of the artificial image, and calculatinga degree of artificiality of the subject pixel by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality.

According to another embodiment of the present invention, there isprovided a program allowing a computer to execute processing includingthe steps of extracting broad-range features including a plurality oftypes of broad-range features from pixels located in a predeterminedarea in relation to a subject pixel of a first image, calculating, in amultidimensional space represented by the plurality of types ofbroad-range features included in the extracted broad-range features,from a positional relationship of the plurality of types of broad-rangefeatures according to a statistical distribution range of an artificialimage, which exhibits a small number of grayscale levels and distinctedges, of the first image, a broad-range degree of artificialityrepresenting a degree by which the plurality of types of broad-rangefeatures belong to the statistical distribution range of the artificialimage, extracting narrow-range features including a plurality of typesof narrow-range features from pixels located in the predetermined areain relation to the subject pixel of the first image, calculating, in amultidimensional space represented by the plurality of types ofnarrow-range features included in the extracted narrow-range features,from a positional relationship of the plurality of types of narrow-rangefeatures according to a statistical distribution range of the artificialimage of the first image, a narrow-range degree of artificialityrepresenting a degree by which the plurality of types of narrow-rangefeatures belong to the statistical distribution range of the artificialimage, and calculating a degree of artificiality of the subject pixel bycombining the broad-range degree of artificiality and the narrow-rangedegree of artificiality.

In the image processing apparatus and method, and the program accordingto an embodiment of the present invention, broad-range featuresincluding a plurality of types of broad-range features are extractedfrom pixels located in a predetermined area in relation to a subjectpixel of a first image. In a multidimensional space represented by theplurality of types of broad-range features included in the extractedbroad-range features, from a positional relationship of the plurality oftypes of broad-range features according to a statistical distributionrange of an artificial image, which exhibits a small number of grayscalelevels and distinct edges, of the first image, a broad-range degree ofartificiality representing a degree by which the plurality of types ofbroad-range features belong to the statistical distribution range of theartificial image is calculated. Narrow-range features including aplurality of types of narrow-range features are extracted from pixelslocated in the predetermined area in relation to the subject pixel ofthe first image. In a multidimensional space represented by theplurality of types of narrow-range features included in the extractednarrow-range features, from a positional relationship of the pluralityof types of narrow-range features according to a statisticaldistribution range of the artificial image of the first image, anarrow-range degree of artificiality representing a degree by which theplurality of types of narrow-range features belong to the statisticaldistribution range of the artificial image is calculated. The degree ofartificiality of the subject pixel is calculated by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality.

The image processing apparatus according to an embodiment of the presentinvention may be an independent apparatus or a block performing imageprocessing.

According to an embodiment of the present invention, the quality of awhole image can be enhanced.

According to an embodiment of the present invention, it is possible toperform optimal processing on an area of natural image components and anarea of artificial image components contained in an image bydistinguishing the two areas from each other.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of the configurationof an image conversion device according to an embodiment of the presentinvention;

FIG. 2 is a flowchart illustrating image conversion processing performedby the image conversion device shown in FIG. 1;

FIG. 3 is a block diagram illustrating the detailed configuration of anatural-image prediction unit;

FIG. 4 illustrates an example of the tap structure of class taps;

FIG. 5 illustrates an example of the tap structure of prediction taps;

FIG. 6 is a flowchart illustrating details of natural-image predictionprocessing;

FIG. 7 is a block diagram illustrating the configuration of a learningdevice;

FIG. 8 illustrates the positional relationship between pixels of asupervisor image and pixels of a learner image;

FIG. 9 is a flowchart illustrating an example of learning processing;

FIG. 10 is a block diagram illustrating the configuration of anartificial-image prediction unit;

FIG. 11 is a block diagram illustrating the detailed configuration of aclassification portion;

FIG. 12 illustrates another example of the tap structure of class taps;

FIG. 13 is a block diagram illustrating the detailed configuration of aprediction portion;

FIG. 14 illustrates another example of the tap structure of predictiontaps;

FIG. 15 is a flowchart illustrating artificial-image predictionprocessing;

FIG. 16 is a flowchart illustrating classification processing;

FIG. 17 is a block diagram illustrating the configuration of anotherlearning device;

FIG. 18 is a block diagram illustrating the detailed configuration of agenerator;

FIG. 19 is a flowchart illustrating another example of learningprocessing;

FIG. 20 is a block diagram illustrating the configuration of anatural-image/artificial-image determining unit;

FIG. 21 is a block diagram illustrating an example of the configurationof a broad edge parameter (BEP) extracting portion;

FIG. 22 is a block diagram illustrating an example of the configurationof a broad flat parameter (BFP) extracting portion;

FIG. 23 is a block diagram illustrating an example of the configurationof a (primary narrow discrimination parameter (PNDP) extracting portion;

FIG. 24 is a block diagram illustrating an example of the configurationof a secondary narrow discrimination parameter (SNDP) extractingportion;

FIG. 25 is a flowchart illustrating natural-image/artificial-imagedetermination processing performed by the natural-image/artificial-imagedetermining unit shown in FIG. 20;

FIG. 26 is a flowchart illustrating BEP extraction processing performedby the BEP extracting portion shown in FIG. 21;

FIG. 27 illustrates reference pixels;

FIG. 28 is a flowchart illustrating BFP extraction processing performedby the BFP extracting portion shown in FIG. 22;

FIG. 29 illustrates the relationship among reference pixels to determinethe adjacent-pixel difference absolute values therebetween;

FIG. 30 illustrates transform function f;

FIG. 31 is a flowchart illustrating broad-range degree-of-artificialitycalculation processing;

FIG. 32 illustrates broad-range boundaries;

FIG. 33 is a flowchart illustrating PNDP extraction processing performedby the PNDP extracting portion shown in FIG. 23;

FIG. 34 illustrates a long tap;

FIG. 35 is a flowchart illustrating SNDP extraction processing performedby the SNDP extracting portion shown in FIG. 24;

FIG. 36 illustrates short taps;

FIG. 37 is a flowchart illustrating narrow-range degree-of artificialitycalculation processing;

FIG. 38 illustrates narrow-range boundaries;

FIG. 39 is a block diagram illustrating another example of theconfiguration of a BEP extracting portion;

FIG. 40 is a flowchart illustrating BEP extraction processing performedby the BEP extracting portion shown in FIG. 39;

FIG. 41 illustrates BEP extraction processing performed by the BEPextracting portion shown in FIG. 39;

FIG. 42 is a block diagram illustrating another example of theconfiguration of the BFP extracting portion;

FIG. 43 is a flowchart illustrating BFP extraction processing performedby the BFP extracting portion shown in FIG. 42;

FIG. 44 illustrates BFP extraction processing performed by the BFPextracting portion shown in FIG. 42;

FIG. 45 is a block diagram illustrating another example of theconfiguration of the PNDP extracting portion;

FIG. 46 is a flowchart illustrating PNDP extracting portion performed bythe PNDP extracting portion shown in FIG. 45;

FIG. 47 is a block diagram illustrating another example of theconfiguration of an SNDP extracting portion;

FIG. 48 is a flowchart illustrating SNDP extraction processing performedby the SNDP extracting portion shown in FIG. 47;

FIG. 49 illustrates transform function F;

FIG. 50 illustrates pixels to be interpolated;

FIG. 51 illustrates another example of the configuration of an imageconversion device;

FIG. 52 is a flowchart illustrating image conversion processingperformed by the image conversion device shown in FIG. 51;

FIGS. 53A, 53B, and 53C illustrate broad-range artificial imageboundaries and broad-range natural image boundaries;

FIGS. 54A, 54B, and 54C illustrate narrow-range artificial imageboundaries and narrow-range natural image boundaries; and

FIG. 55 is a block diagram illustrating the configuration of a personalcomputer implementing the image conversion device.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Before describing an embodiment of the present invention, thecorrespondence between the features of the claims and the embodimentdisclosed in the present invention is discussed below. This descriptionis intended to assure that the embodiment supporting the claimedinvention is described in this specification. Thus, even if an elementin the following embodiment is not described as relating to a certainfeature of the present invention, that does not necessarily mean thatthe element does not relate to that feature of the claims. Conversely,even if an element is described herein as relating to a certain featureof the claims, that does not necessarily mean that the element does notrelate to other features of the claims.

Furthermore, this description should not be construed as restrictingthat all the aspects of the invention disclosed in the embodiment aredescribed in the claims. That is, the description does not deny theexistence of aspects of the present invention that are described in theembodiment but not claimed in the invention of this application, i.e.,the existence of aspects of the present invention that in future may beclaimed by a divisional application, or that may be additionally claimedthrough amendments.

An image processing apparatus according to an embodiment of the presentinvention includes broad-range feature extraction means (e.g., abroad-range feature extracting portion 911 shown in FIG. 20) forextracting broad-range features including a plurality of types ofbroad-range features from pixels located in a predetermined area inrelation to a subject pixel of a first image, broad-rangedegree-of-artificiality calculation means (e.g., a broad-rangedegree-of-artificiality calculator 912 shown in FIG. 20) forcalculating, in a multidimensional space represented by the plurality oftypes of broad-range features included in the broad-range featuresextracted by the broad-range feature extraction means, from a positionalrelationship of the plurality of types of broad-range features accordingto a statistical distribution range of an artificial image, whichexhibits a small number of grayscale levels and distinct edges, of thefirst image, a broad-range degree of artificiality representing a degreeby which the plurality of types of broad-range features belong to thestatistical distribution range of the artificial image, narrow-rangefeature extraction means (e.g., a narrow-range feature extractingportion 914 shown in FIG. 20) for extracting narrow-range featuresincluding a plurality of types of narrow-range features from pixelslocated in the predetermined area in relation to the subject pixel ofthe first image, narrow-range degree-of-artificiality calculation means(e.g., a narrow-range degree-of-artificiality calculator 915 shown inFIG. 20) for calculating, in a multidimensional space represented by theplurality of types of narrow-range features included in the narrow-rangefeatures extracted by the narrow-range feature extraction means, from apositional relationship of the plurality of types of narrow-rangefeatures according to a statistical distribution range of the artificialimage of the first image, a narrow-range degree of artificialityrepresenting a degree by which the plurality of types of narrow-rangefeatures belong to the statistical distribution range of the artificialimage, and degree-of-artificiality calculation means (e.g., adegree-of-artificiality calculator 913 shown in FIG. 20) for calculatinga degree of artificiality of the subject pixel by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality.

The image processing apparatus may further include first predictionmeans (e.g., an artificial-image prediction unit 132 shown in FIG. 1)for predicting, from the first image, a second image which is obtainedby increasing the quality of the artificial image, second predictionmeans (e.g., a natural-image prediction unit 131 shown in FIG. 1) forpredicting, from the first image, a third image which is obtained byincreasing the quality of a natural image exhibiting a large number ofgrayscale levels and indistinct edges, and synthesizing means (e.g., asynthesizer 133 shown in FIG. 1) for combining the second image and thethird image on the basis of the degree of artificiality.

The first prediction means may include first classification means (e.g.,a classification portion 651 shown in FIG. 10) for classifying pixels ofthe second image into first classes, first storage means (e.g., aprediction coefficient memory 654 shown in FIG. 10) for storing a firstprediction coefficient for each of the first classes, the firstprediction coefficient being obtained by conducting learning by using aplurality of artificial images, and first computation means (e.g., aprediction portion 655 shown in FIG. 10) for determining, from the firstimage, the second image having a higher quality than the first image byperforming computation using the first image and the first predictioncoefficients for the first classes into which the pixels of the secondimage are classified. The second prediction means may include secondclassification means (e.g., a class tap extracting portion 551 or anadaptive dynamic range coding (ADRC) processor 552 shown in FIG. 3) forclassifying pixels of the third image into second classes, secondstorage means (e.g., a prediction coefficient memory 555 shown in FIG.3) for storing a second prediction coefficient for each of the secondclasses, the second prediction coefficient being obtained by conductinglearning by using a plurality of natural images, and second computationmeans (e.g., a prediction computation portion 557 shown in FIG. 3) fordetermining the third image from the first image by performingcomputation using the first image and the second prediction coefficientsfor the second classes into which the pixels of the third image areclassified.

The broad-range degree-of-artificiality calculation means (e.g., thebroad-range degree-of-artificiality calculator 912 shown in FIG. 20) mayinclude broad-range artificial-image distribution range storage means(e.g., broad-range boundary memory 953 shown in FIG. 20) for storing thestatistical distribution range of the artificial image of the firstimage in the multidimensional space represented by the plurality oftypes of broad-range features, and the broad-rangedegree-of-artificiality calculation means may calculate the broad-rangedegree of artificiality from the positional relationship of theplurality of types of broad-range features extracted by the broad-rangefeature extraction means according to the statistical distribution rangeof the artificial image in the multidimensional space stored in thebroad-range artificial-image distribution range storage means.

The narrow-range degree-of-artificiality calculation means (e.g., thenarrow-range degree-of-artificiality calculator 915 shown in FIG. 20)may include narrow-range artificial-image distribution range storagemeans (e.g., a narrow-range boundary memory 993 shown in FIG. 20) forstoring the statistical distribution range of the artificial image ofthe first image in the multidimensional space represented by theplurality of types of narrow-range features, and the narrow-rangedegree-of-artificiality calculation means may calculate the narrow-rangedegree of artificiality from the positional relationship of theplurality of types of narrow-range features extracted by thenarrow-range feature extraction means according to the statisticaldistribution range of the artificial image in the multidimensional spacestored in the narrow-range artificial-image distribution range storagemeans.

The broad-range feature extraction means may include edge featureextraction means (e.g., a BEP extracting portion 931 shown in FIG. 20)for extracting a feature representing the presence of an edge from thepixels located in the predetermined area, and flat feature extractionmeans (e.g., a BFP extracting portion 932 shown in FIG. 20) forextracting a feature representing the presence of a flat portion fromthe pixels located in the predetermined area.

The edge feature extraction means (e.g., the BEP extracting portion 931shown in FIG. 21) may extract, as the feature representing the presenceof an edge, a difference dynamic range of the pixels located in thepredetermined area by using a difference of pixel values between thesubject pixel and each of the pixels located in the predetermined area.

The edge feature extraction means (e.g., the BEP extracting portion 931shown in FIG. 21) may extract, as the feature representing the presenceof an edge, a difference dynamic range of the pixels located in thepredetermined area by using a difference of the pixel values between thesubject pixel and each of the pixels located in the predetermined areaafter applying a weight to the difference of the pixel values inaccordance with a distance therebetween.

The edge feature extraction means (e.g., the BEP extracting portion 931shown in FIG. 39) may extract, as the feature representing the presenceof an edge, a predetermined order of higher levels of differenceabsolute values between adjacent pixels of the pixels located in thepredetermined area.

The edge feature extraction means (e.g., the BEP extracting portion 931shown in FIG. 39) may extract, as the feature representing the presenceof an edge, the average of higher first through second levels ofdifference absolute values between adjacent pixels of the pixels locatedin the predetermined area or the sum of the higher first through secondlevels of the difference absolute values after applying a weight to eachof the difference absolute values.

The flat feature extraction means (e.g., the BFP extracting portion 932shown in FIG. 22) may extract, as the feature representing the presenceof a flat portion, the number of difference absolute values betweenadjacent pixels of the pixels located in the predetermined area whichare smaller than a predetermined threshold.

The flat feature extraction means (e.g., the BFP extracting portion 932shown in FIG. 22) may extract, as the feature representing the presenceof a flat portion, the sum of difference absolute values betweenadjacent pixels of the pixels located in the predetermined area aftertransforming the difference absolute values by a predetermined function.

The flat feature extraction means (e.g., the BFP extracting portion 932shown in FIG. 22) may extract, as the feature representing the presenceof a flat portion, the sum of difference absolute values between theadjacent pixels of the pixels located in the predetermined area aftertransforming the difference absolute values by a predetermined functionand after applying a weight to each of the transformed differenceabsolute values in accordance with a distance from the subject pixel toeach of the pixels located in the predetermined area.

The flat feature extraction means (e.g., the BFP extracting portion 932shown in FIG. 42) may extract, as the feature representing the presenceof a flat portion, a predetermined order of lower levels of differenceabsolute values between adjacent pixels of the pixels located in thepredetermined area.

The flat feature extraction means (e.g., the BFP extracting portion 932shown in FIG. 42) may extract, as the feature representing the presenceof a flat portion, the average of lower first through second levels ofdifference absolute values between adjacent pixels of the pixels locatedin the predetermined area or the sum of the lower first through secondlevels of the difference absolute values after applying a weight to eachof the difference absolute values.

The narrow-range feature extraction (e.g., the narrow-range featureextracting portion 914 shown in FIG. 20) means may extract, from thepixels located in the predetermined area, the narrow-range featuresincluding two types of features selected from features representing thinlines, edges, points, flat portions in the vicinity of edges, andgradation.

The narrow-range feature extraction means (e.g., the narrow-rangefeature extracting portion 914 shown in FIG. 20) may include firstnarrow-range feature extraction means (e.g., a PNDP extracting portion971 shown in FIG. 20) for extracting, as a first feature of thenarrow-range features, a pixel-value dynamic range obtained bysubtracting a minimum pixel value from a maximum pixel value of pixelslocated in a first area included in the predetermined area, and secondnarrow-range feature extraction means (e.g., an SNDP extracting portion972 shown in FIG. 20) for extracting, as a second feature of thenarrow-range features, a pixel-value dynamic range obtained bysubtracting a minimum pixel value from a maximum pixel value of pixelslocated in a second area including the subject pixel and included in thefirst area.

The second narrow-range feature extraction means (e.g., an SNDPextracting portion 972 shown in FIG. 24) may extract, as the secondfeature, a minimum pixel-value dynamic range of pixel-value dynamicranges obtained from a plurality of the second areas.

The first narrow-range feature extraction means (e.g., the PNDPextracting portion 971 shown in FIG. 45) may extract, as the firstfeature of the narrow-range features, a pixel-value dynamic range of thepixel values of the pixels located in the predetermined area. The secondnarrow-range feature extraction means (e.g., the SNDP extracting portion972 shown in FIG. 47) may extract, as the second feature of thenarrow-range features, the sum of difference absolute values between thesubject pixel and the pixels located in the predetermined area aftertransforming the difference absolute values by a predetermined functionand after applying a weight to each of the transformed differenceabsolute values.

The second narrow-range feature extraction means (e.g., the SNDPextracting portion 972 shown in FIG. 47) may include weight calculationmeans (e.g., a weight calculator 1165 shown in FIG. 47) for calculatingthe weight in accordance with the sum of the difference absolute valuesbetween adjacent pixels of all the pixels located on a path from thesubject pixel to the pixels located in the first area.

According to another embodiment of the present invention, there isprovided an image processing method including the steps of extractingbroad-range features including a plurality of types of broad-rangefeatures from pixels located in a predetermined area in relation to asubject pixel of a first image (e.g., steps S831 and S832 in theflowchart in FIG. 25), calculating, in a multidimensional spacerepresented by the plurality of types of broad-range features includedin the extracted broad-range features, from a positional relationship ofthe plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image (e.g., step S834 in theflowchart in FIG. 25), extracting narrow-range features including aplurality of types of narrow-range features from pixels located in thepredetermined area in relation to the subject pixel of the first image(e.g., steps S835 and S836 in the flowchart in FIG. 25), calculating, ina multidimensional space represented by the plurality of types ofnarrow-range features included in the extracted narrow-range features,from a positional relationship of the plurality of types of narrow-rangefeatures according to a statistical distribution range of the artificialimage of the first image, a narrow-range degree of artificialityrepresenting a degree by which the plurality of types of narrow-rangefeatures belong to the statistical distribution range of the artificialimage (e.g., step S838 in the flowchart in FIG. 25), and calculating adegree of artificiality of the subject pixel by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality (e.g., step S839 in the flowchart in FIG. 25).

Embodiments of the present invention are described in detail below withreference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an image conversion device 101according to an embodiment of the present invention. The imageconversion device 101 includes a cyclic interlace/progressive (IP)converter 111, an output phase converter 112, an image processor 113,and a natural-image/artificial-image determining unit 114. The cyclic IPconverter 111 includes an IP converter 121 and a cyclic converter 122.The image processor 113 includes a natural-image prediction unit 131, anartificial-image prediction unit 132, and a synthesizer 133.

An interlace SD image to be processed is input into the IP converter 121and the cyclic converter 122 of the cyclic IP converter 111.

The IP converter 121 converts the input interlace SD image (hereinafteralso referred to as an “input image”) into a progressive SD image(hereinafter also referred to as an “intermediate image”) according to apredetermined method, and supplies the converted progressive SD image tothe cyclic converter 122.

The cyclic converter 122 determines motion vectors between the inputimage and the progressive SD image of the previous frame (one framebefore) output from the cyclic converter 122 (such an image is alsoreferred to as an “output image”). The cyclic converter 122 then addsthe pixel values of the output image motion-compensated based on thedetermined motion vectors to the pixel values of the input image byusing cyclic coefficients as weights, thereby improving the intermediateimage. That is, the cyclic converter 122 converts the intermediate imageinto an output image, which is a progressive SD image of a qualityhigher than the intermediate image, and supplies the resulting outputimage to the output phase converter 112. The cyclic coefficients are setbased on whether each pixel of the intermediate image exists in theoriginal input image and also based on the magnitudes of the motionvectors in the vertical direction and the reliabilities representing theprobabilities of the motion vectors.

The output phase converter 112 interpolates the SD image having a firstpixel number supplied from the cyclic converter 122 in the horizontaland vertical directions to generate an HD image having a second pixelnumber. The second pixel number is greater than the first pixel number.The output phase converter 112 supplies the HD image to thenatural-image prediction unit 131, the artificial-image prediction unit132, and the natural-image/artificial-image determining unit 114.

The image processor 113 performs processing for converting the HD imageinto a high-quality image based on the degrees of artificiality suppliedfrom the natural-image/artificial-image determining unit 114, andoutputs the high-quality HD image.

The natural-image/artificial-image determining unit 114 determines foreach pixel of the HD image supplied from the output phase converter 112whether it belongs to an artificial image area or a natural image area,and outputs determination results to the image processor 113 as thedegrees of artificiality. That is, the degree of artificialityrepresents the ratio of artificial image components to natural imagecomponents in an intermediate area, which is between the artificialimage area and the natural image area, by a value from 0 to 1.

The natural-image prediction unit 131 predicts, from the HD imagesupplied from the output phase converter 112, a high-quality HD imagewhich can be obtained by increasing the quality of natural imagecomponents contained in the input HD image (such a high-quality HD imageis hereafter referred to as a “high-quality natural image”). Morespecifically, in accordance with the features of the input HD image, thenatural-image prediction unit 131 allocates the subject pixels intoclasses optimal for the features of the natural image. Then, thenatural-image prediction unit 131 performs computation by using theinput HD image and prediction coefficients corresponding to the classes,which are used for predicting the high-quality natural image, to predictthe high-quality natural image from the input HD image. Thenatural-image prediction unit 131 supplies the computed high-qualitynatural image to the synthesizer 133.

As in the natural-image prediction unit 131, the artificial-imageprediction unit 132 predicts, from the HD image supplied from the outputphase converter 112, a high-quality HD image which can be obtained byincreasing the quality of artificial image components contained in theinput HD image (such a high-quality HD image is hereafter referred to asa “high-quality artificial image”). More specifically, in accordancewith the features of the input HD image, the artificial-image predictionunit 132 allocates the pixels forming the high-quality artificial imageto be determined from the input HD image into classes optimal for thefeatures of the artificial image. Then, the artificial-image predictionunit 132 performs calculations by using the input HD image andprediction coefficients corresponding to the classes, which are used forpredicting the high-quality artificial image, to predict thehigh-quality artificial image from the input HD image. Theartificial-image prediction unit 132 supplies the calculatedhigh-quality artificial image to the synthesizer 133.

The synthesizer 133 combines, based on the determination resultssupplied from the natural-image/artificial-image determining unit 114,the pixel values of the pixels forming the high-quality natural imagesupplied from the natural-image prediction unit 131 with the pixelvalues of the pixels forming the high-quality artificial image suppliedfrom the artificial-image prediction unit 132 in accordance with thedegrees of artificiality of the individual pixels. The synthesizer 133then outputs the synthesized HD image.

The image conversion processing executed by the image conversion device101 is described below with reference to the flowchart in FIG. 2. Thisprocessing is started, for example, when an interlace SD image is inputfrom an external source into the image conversion device 101.

In step S1, the IP converter 121 performs IP conversion. Morespecifically, the IP converter 121 converts an interlace input imageinto a progressive intermediate image according to a predeterminedmethod, and supplies the IP-converted intermediate image to the cyclicconverter 122.

In step S2, the cyclic converter 122 performs cyclic conversionprocessing. More specifically, the cyclic converter 122 determinesmotion vectors between the input image and the output image of theprevious frame (one frame before) output from the cyclic converter 122.The cyclic converter 122 then adds the pixel values of themotion-compensated output image based on the determined motion vectorsto the pixel values of the input image by using cyclic coefficients asweights, thereby improving the intermediate image. The cyclic converter122 converts the intermediate image into an output image, which is aprogressive SD image of a quality higher than the intermediate image,and supplies the resulting output image to the output phase converter112.

In step S3, the output phase converter 112 performs output phaseconversion processing. More specifically, the output phase converter 112interpolates the SD image supplied from the cyclic converter 122 in thehorizontal and vertical directions to generate an HD image. The outputphase converter 112 then supplies the HD image to the natural-imageprediction unit 131, the artificial-image prediction unit 132, and thenatural-image/artificial-image determining unit 114.

In step S4, the natural-image prediction unit 131 performs natural-imageprediction processing. According to this processing, a high-qualitynatural image is predicted from the HD image and is supplied to thesynthesizer 133. Details of the natural-image prediction processing arediscussed below with reference to FIG. 6.

In step S5, the artificial-image prediction unit 132 performsartificial-image prediction processing. According to this processing, ahigh-quality artificial image is predicted from the HD image and issupplied to the synthesizer 133. Details of the artificial-imageprediction processing are discussed below with reference to FIG. 15.

In step S6, the natural-image/artificial-image determining unit 114performs natural-image/artificial-image determination processing.According to this processing, the natural-image/artificial-imagedetermining unit 114 determines whether each pixel of the HD imagesupplied from the output phase converter 112 belongs to an artificialimage area or a natural image area, and outputs determination results tothe synthesizer 133 as the degrees of artificiality. Details of thenatural-image/artificial-image determination processing are discussedbelow with reference to FIG. 25.

In step S7, the synthesizer 133 synthesizes an image. More specifically,the synthesizer 133 combines, based on determination results suppliedfrom the natural-image/artificial-image determining unit 114, the pixelvalues of the pixels forming the high-quality natural image suppliedfrom the natural-image prediction unit 131 with the pixel values of thepixels forming the high-quality artificial image supplied from theartificial-image prediction unit 132 in accordance with the degrees ofartificiality of the individual pixels. The synthesizer 133 outputs thesynthesized HD image to a subsequent device.

If the image conversion processing is continuously performed on aplurality of images, steps S1 through S7 are repeated.

FIG. 3 is a block diagram illustrating the configuration of thenatural-image prediction unit 131 shown in FIG. 1.

The natural-image prediction unit 131 includes a class tap extractingportion 551, an adaptive dynamic range coding (ADRC) processor 552, acoefficient seed memory 553, a prediction coefficient generator 554, aprediction coefficient memory 555, a prediction tap extracting portion556, and a prediction computation portion 557. The natural-imageprediction unit 131 predicts a high-quality natural image from theprogressive HD image supplied from the output phase converter 112 shownin FIG. 1 after removing noise from the natural image.

A progressive HD image supplied from the output phase converter 112shown in FIG. 1 is supplied to the natural-image prediction unit 131,and more specifically, to the class tap extracting portion 551 and theprediction tap extracting portion 556.

The class tap extracting portion 551 sequentially selects the pixelsforming the high-quality natural image determined from the input HDimage as subject pixels, and extracts some of the pixels forming the HDimage as class taps, which are used for classifying the subject pixels.The class tap extracting portion 551 then supplies the extracted classtaps to the ADRC processor 552.

The ADRC processor 552 performs ADRC processing on the pixel values ofthe pixels forming the class taps supplied from the class tap extractingportion 551 to detect the ADRC code as the feature of the waveform ofthe class taps.

In K-bit ADRC processing, the maximum value MAX and the minimum valueMIN of the pixel values of the pixels forming the class taps aredetected, and DR=MAX−MIN is set as the local dynamic range of a set, andthen, the pixel values of the pixels forming the class taps arere-quantized into K bits based on the dynamic range. That is, theminimum value MIN is subtracted from the pixel value of each pixelforming the class taps and the resulting value is divided by DR/2^(K).

Then, the K-bit pixel values of the pixels forming the class taps arearranged in a predetermined order, resulting in a bit string, which isthen output as the ADRC code. Accordingly, if one-bit ADRC processing isperformed on the class taps, the pixel value of each pixel forming theclass taps is divided by the average of the maximum value MAX and theminimum value MIN so that it is re-quantized into one bit with thedecimal fractions omitted. That is, the pixel value of each pixel isbinarized. Then, a bit string of the one-bit pixel values arranged in apredetermined order is output as the ADRC code.

The ADRC processor 552 determines the class based on the detected ADRCcode to classify each subject pixel, and then supplies the determinedclass to the prediction coefficient memory 555. For example, the ADRCprocessor 552 directly supplies the ADRC code to the predictioncoefficient memory 555 as the class.

The coefficient seed memory 553 stores a coefficient seed, which isobtained by learning discussed below with reference to FIGS. 7 through9, for each class. The prediction coefficient generator 554 reads acoefficient seed from the coefficient seed memory 553. The predictioncoefficient generator 554 then generates a prediction coefficient fromthe read coefficient seed by using a polynomial containing a parameter hand a parameter v, which are input by a user, for determining thehorizontal resolution and the vertical resolution, respectively, andsupplies the generated prediction coefficient to the predictioncoefficient memory 555.

The prediction coefficient memory 555 reads out the predictioncoefficient according to the class supplied from ADRC processor 552, andsupplies the read prediction coefficient to the prediction computationportion 557.

The prediction taps and the class taps may have the same tap structureor different tap structures.

The prediction tap extracting portion 556 extracts, from the input HDimage, as prediction taps, some of the pixels forming the HD image usedfor predicting the pixel value of a subject pixel. The prediction tapextracting portion 556 supplies the extracted prediction taps to theprediction computation portion 557.

The prediction computation portion 557 performs prediction computation,such as linear expression computation, for determining the predictionvalue of the true value of the subject pixel by using the predictiontaps supplied from the prediction tap extracting portion 556 and theprediction coefficient supplied from the prediction coefficient memory555. Then, the prediction computation portion 557 predicts the pixelvalue of the subject pixel, i.e., the pixel value of a pixel forming thehigh-quality natural image, and outputs the predicted pixel value to thesynthesizer 133.

FIG. 4 illustrates an example of the tap structure of class tapsextracted by the class tap extracting portion 551 shown in FIG. 3.

In FIG. 4, among the pixels forming the HD image supplied from theoutput phase converter 112, the white circles indicate the pixelsforming the class taps, the circles represented by broken curvesrepresent the pixels that do not form the class taps, and the blackcircle designates the subject pixel. The same applies to FIG. 5.

In FIG. 4, nine pixels form the class taps. More specifically, around apixel p64 forming the HD image corresponding to a subject pixel q6, fivepixels p60, p61, p64, p67, and p68 aligned every other pixel in thevertical direction, and four pixels p62, p63, p65, and p66 aligned everyother pixel, except for the pixel p64, in the horizontal direction, aredisposed as the class taps, i.e., a so-called “cross-shaped” class tapstructure is formed.

FIG. 5 illustrates an example of the tap structure of prediction tapsextracted by the prediction tap extracting portion 556 shown in FIG. 3.

In FIG. 5, 13 pixels form the prediction taps. More specifically, amongthe pixels forming the HD pixel supplied from the output phase converter112, around a pixel p86 forming the HD image corresponding to a subjectpixel q8, five pixels p80, p82, p86, p90, and p92 aligned every otherpixel in the vertical direction, four pixels p84, p85, p87, and p88aligned every other pixel, except for the pixel p86, in the verticaldirection, two pixels p81 and p89 aligned every other pixel, except forthe pixel p85, in the vertical direction around the pixel p85, and twopixels p83 and p91 aligned every other pixel, except for the pixel p87,in the vertical direction around the pixel p87, are disposed as theprediction taps, i.e., a generally rhomboid prediction tap structure isformed.

In FIGS. 4 and 5, the nine pixels p60 through p68 forming the class tapsand the 13 pixels p80 through p92 forming the prediction taps,respectively, are arranged in the vertical direction or in thehorizontal direction every other pixel, i.e., at regular intervals oftwo pixels. However, the intervals of the pixels forming the class tapsor the prediction taps are not restricted to two pixels, and may bechanged in accordance with the ratio of the number of pixels of theconverted HD image to the number of pixels of the SD image beforeconversion, i.e., the interpolation factor, employed in the output phaseconverter 112.

It is now assumed, for example, that the output phase converter 112converts the SD image so that the numbers of pixels in the horizontaland vertical directions are doubled. In this case, if class taps orprediction taps are formed of the pixels arranged at intervals of twopixels in the horizontal or vertical direction, as shown in FIG. 4 or 5,either of the interpolated pixels or the pixels that are notinterpolated can form the class taps or the prediction taps. Thus, theprecision of the prediction processing performed by the natural-imageprediction unit 131 can be improved compared to that, for example, inthe case where both the interpolated pixels and the pixels that are notinterpolated form class taps or prediction taps, i.e., the class taps orthe prediction taps could be arranged adjacent to each other.

Details of the natural-image prediction processing in step S4 in FIG. 2performed by the natural-image prediction unit 131 shown in FIG. 3 arediscussed below with reference to FIG. 6.

In step S551, the class tap extracting portion 551 selects, as a subjectpixel, one of the pixels forming the high-quality natural imagedetermined from the HD image supplied from the output phase converter112 shown in FIG. 1.

In step S552, the class tap extracting portion 551 then extracts, asclass taps, some of the pixels forming the input HD image, such as thoseshown in FIG. 4, used for classifying the subject pixel selected in stepS551, and supplies the extracted class taps to the ADRC processor 552.

In step S553, the ADRC processor 552 performs ADRC processing on thepixel values of the pixels forming the class taps supplied from theclass tap extracting portion 551, and sets the resulting ADRC code asthe feature of the class taps.

In step S554, the ADRC processor 552 determines the class based on theADRC code to classify the subject pixel, and then supplies thedetermined class to the prediction coefficient memory 555.

In step S555, the prediction coefficient generator 554 reads out thecorresponding coefficient seed from the coefficient seed memory 553.

In step S556, the prediction coefficient generator 554 generates theprediction coefficient from the coefficient seed read from thecoefficient seed memory 553 by using the polynomial containing theparameters h and v input by the user, and supplies the generatedprediction coefficient to the prediction coefficient memory 555. Detailsof the processing for generating a prediction coefficient from acoefficient seed are discussed below.

In step S557, the prediction coefficient memory 555 reads out theprediction coefficient on the basis of the class supplied from the ADRCprocessor 552, and supplies the read prediction coefficient to theprediction computation portion 557.

In step S558, the prediction tap extracting portion 556 extracts, asprediction taps, some of the pixels forming the input HD image, such asthose shown in FIG. 5, used for predicting the pixel value of thesubject pixel. The prediction tap extracting portion 556 supplies theextracted prediction taps to the prediction computation portion 557.

In step S559, the prediction computation portion 557 performs predictioncomputation, for example, linear expression computation, for determiningthe prediction value of the true value of the subject pixel by using theprediction taps supplied from the prediction tap extracting portion 556and the prediction coefficient supplied from the prediction coefficientmemory 555.

In step S560, the prediction computation portion 557 outputs thepredicted pixel value of the subject pixel as a result of the predictioncomputation, i.e., the pixel value of the corresponding pixel formingthe high-quality natural image, to the synthesizer 133.

In step S561, the class tap extracting portion 551 determines whetherall the pixels forming the high-quality natural image determined fromthe input HD image have been selected as the subject pixels. If it isdetermined in step S561 that not all the pixels forming the high-qualitynatural image have been selected as the subject pixels, the processproceeds to step S562. In step S562, the class tap extracting portion551 selects a pixel which has not been selected as the subject pixel,and returns to step S552. Steps S552 and the subsequent steps are thenrepeated. If it is determined in step S561 that all the pixels formingthe high-quality natural image have been selected as the subject pixels,the natural-image prediction processing is completed.

As discussed above, the natural-image prediction unit 131 predicts ahigh-quality natural image from the HD image supplied from the outputphase converter 112 and outputs the predicted high-quality naturalimage. That is, the natural-image prediction unit 131 converts the HDimage into the high-quality natural image and outputs it.

As described above, in the image conversion device 101 shown in FIG. 1,the output phase converter 112 converts an SD image supplied from thecyclic converter 122 into an HD image, and then supplies the convertedHD image to the natural-image prediction unit 131. Accordingly, thenumber of pixels forming the image after prediction is the same as thatbefore prediction, and the positions of the pixels forming the imageafter prediction are not displaced from those of the pixels forming theimage before prediction.

Accordingly, the natural-image prediction unit 131 can predict the pixelvalue of a subject pixel of the high-quality natural image by using theprediction taps formed of the pixels of the HD image which are in phasewith the subject pixel. As a result, the natural-image prediction unit131 can accurately predict the high-quality natural image to performhigh-precision image conversion. That is, the output phase converter 112and the natural-image prediction unit 131 can accurately convert an SDimage supplied from the cyclic converter 122 into a high-quality naturalimage, which is a high-quality HD image having the number of pixelsdifferent from that of the SD image.

Additionally, the natural-image prediction unit 131 determines thefeature of the waveform of the pixels forming the class taps, and thenclassifies the subject pixel by using the determined feature.Accordingly, the subject pixel can be suitably classified according tothe feature of a natural image having relatively a small number of flatportions. As a result, the natural-image prediction unit 131 can enhancethe quality of the natural image components contained in the HD image.

A description is now given of the prediction computation performed bythe prediction computation portion 557 shown in FIG. 3 and learning forprediction coefficients used for the prediction computation.

It is now assumed that linear prediction computation is conducted aspredetermined prediction computation for predicting the pixel value of apixel forming a high-quality natural image (hereinafter such a pixel issometimes referred to as a “high-quality natural image pixel”) by usingprediction taps extracted from an input HD image and a predictioncoefficient. In this case, the pixel value y of the high-quality naturalimage pixel can be determined by the following linear expression:

$\begin{matrix}{y = {\sum\limits_{n = 1}^{N}{W_{n}x_{n}}}} & (1)\end{matrix}$where x_(n) represents the pixel value of the n-th pixel of the HD image(hereinafter sometimes referred to as an “HD image pixel”) forming theprediction taps for the high-quality natural image pixel having thepixel value y, and W_(n) designates the n-th prediction coefficient tobe multiplied by the n-th pixel value of the HD image. It should benoted that the prediction taps are formed of N HD image pixels x₁, x₂, .. . , and x_(n) in equation (1).

The pixel value y of the high-quality natural image pixel may bedetermined from a higher-order expression instead of the linearexpression represented by equation (1).

If the true value of the pixel value of the k-sample high-qualitynatural image pixel is represented by y_(k) and the prediction value ofthe true value y_(k) obtained by equation (1) is represented by y_(k)′,the prediction error e_(k) can be expressed by the following equation.e _(k) =y _(k) −y _(k)′  (2)

The prediction value y_(k)′ in equation (2) can be obtained by equation(1). Accordingly, if equation (1) is substituted into equation (2), thefollowing equation can be found:

$\begin{matrix}{e_{k} = {y_{k} - ( {\sum\limits_{n = 1}^{N}{W_{n}x_{n,k}}} )}} & (3)\end{matrix}$where x_(n,k) designates the n-th HD image pixel forming the predictiontaps for the k-sample high-quality natural image pixel.

The prediction coefficient W_(n) that reduces the prediction error e_(k)in equation (3) or (2) to 0 or statistically minimizes the predictionerror e_(k) is the optimal prediction coefficient W_(n) for predictingthe high-quality natural image pixel. Generally, however, it isdifficult to obtain such a prediction coefficient W_(n) for allhigh-quality natural image pixels.

If, for example, the method of least squares, is employed as thestandard for representing that the prediction coefficient W_(n) isoptimal, the optimal prediction coefficient W_(n) can be obtained byminimizing the total error E of square errors expressed by the followingequation:

$\begin{matrix}{E = {\sum\limits_{k = 1}^{K}e_{k}^{2}}} & (4)\end{matrix}$where K is the number of samples of sets of the true values y_(k) of thepixel values of a high-quality image and the HD image pixels x_(1,k),x_(2,k), . . . , x_(N,k) forming the prediction taps for the true valuesy_(k), i.e., the number of samples for conducting learning.

The minimum value of the total error E of the square errors in equation(4) can be given by the prediction coefficient W_(n) that allows thevalue obtained by partially differentiating the total error E withrespect to the prediction coefficient W_(n) to be 0, as expressed byequation (5).

$\begin{matrix}{{\frac{\partial E}{\partial W_{n}} = {{{e_{1}\frac{\partial e_{1}}{\partial W_{n}}} + {e_{n}\frac{\partial e_{2}}{\partial W_{n}}} + \ldots + {e_{k}\frac{\partial e_{k\; 2}}{\partial W_{n}}}} = 0}}( {{n = 1},2,\ldots\mspace{11mu},N} )} & (5)\end{matrix}$

Then, if equation (3) is partially differentiated with respect to theprediction coefficient W_(n), the following equation can be found.

$\begin{matrix}{{\frac{\partial e_{k}}{\partial W_{1}} = {- x_{1,k}}},{\frac{\partial e_{k}}{\partial W_{2}} = {- x_{2,k}}},\ldots\mspace{11mu},} & (6) \\{{\frac{\partial e_{k}}{\partial W_{N}} = {- x_{N,k}}},( {{k = 1},2,\ldots\mspace{11mu},K} )} & \;\end{matrix}$

The following equation can be found from equations (5) and (6).

$\begin{matrix}{{{\sum\limits_{k = 1}^{K}{e_{k}x_{1,k}}} = 0},{{\sum\limits_{k = 1}^{K}{e_{k}x_{2,k}}} = 0},{{\ldots\mspace{11mu}{\sum\limits_{k = 1}^{K}{e_{k}x_{N,k}}}} = 0}} & (7)\end{matrix}$

By substituting equation (3) into e_(k) in equation (7), equation (7)can be represented by normal equations, as expressed by equation (8).

$\begin{matrix}{{\begin{bmatrix}( {\sum\limits_{k = 1}^{K}{x_{1,k}x_{1,k}}} ) & ( {\sum\limits_{k = 1}^{K}{x_{1,k}x_{2,k}}} ) & \ldots & ( {\sum\limits_{k = 1}^{K}{x_{1,k}x_{N,k}}} ) \\( {\sum\limits_{k = 1}^{K}{x_{2,k}x_{1,k}}} ) & ( {\sum\limits_{k = 1}^{K}{x_{2,k}x_{2,k}}} ) & \ldots & ( {\sum\limits_{k = 1}^{K}{x_{2,k}x_{N,k}}} ) \\\vdots & \vdots & \ddots & \vdots \\( {\sum\limits_{k = 1}^{K}{x_{N,k}x_{1,k}}} ) & ( {\sum\limits_{k = 1}^{K}{x_{N,k}x_{2,k}}} ) & \ldots & ( {\sum\limits_{k = 1}^{K}{x_{N,k}x_{N,k}}} )\end{bmatrix}\lbrack \begin{matrix}W_{1} \\W_{2} \\\vdots \\W_{N}\end{matrix} \rbrack}\begin{matrix} = \\ = \\\; \\ = \end{matrix}} & (8) \\{\mspace{515mu}\lbrack \begin{matrix}( {\sum\limits_{k = 1}^{K}{x_{1,k}y_{k}}} ) \\( {\sum\limits_{k = 1}^{K}{x_{2,k}y_{k}}} ) \\\vdots \\( {\sum\limits_{k = 1}^{K}{x_{N,k}y_{k}}} )\end{matrix} \rbrack} & \;\end{matrix}$

The normal equations in equation (8) can be solved with respect to theprediction coefficient W_(n) by using, for example, a sweeping-outmethod (Gauss-Jordan elimination method).

By solving the normal equations in equation (8) for each class, theoptimal prediction coefficient W_(n) that minimizes the total error E ofthe least squares can be found for each class.

Polynomials used for generating prediction coefficients by theprediction coefficient generator 554 shown in FIG. 3 and learning forcoefficient seeds used for the polynomials are discussed below.

If, for example, a polynomial, is used as an expression for generating aprediction coefficient by using input parameters h and v and acoefficient seed, the prediction coefficient W_(n) for each class andfor each set of the parameters h and v can be found by the followingequation:

$\begin{matrix}{W_{n} = {w_{n,0} + {w_{n,1}v} + {w_{n,2}h} + {w_{n,3}v^{2}} + {w_{n,4}{vh}} + {w_{n,5}h^{2}} + {w_{n,6}v^{3}} + {w_{n,7}v^{2}h} + {w_{n,8}{vh}^{2}} + {w_{n,9}h^{3}}}} & (9)\end{matrix}$where w_(n,k)(k=1, 2, . . . , 9) represents the k-th term coefficientamong the coefficient seeds for generating the n-th predictioncoefficient W_(n) which is to be multiplied by the pixel value x_(n) ofthe n-th pixel of the HD image, the n-th pixel of the HD image formingthe prediction taps for the high-quality natural image pixel having then-th pixel value y expressed by equation (1).

If the true value of the n-th prediction coefficient corresponding tothe parameters h and v is represented by W_(vhn) and if the estimationvalue of the true value W_(vhn) obtained by equation (9) is indicated byW_(vhn)′, the estimation error e_(vhn) can be expressed by the followingequation.e _(vhn) =W _(vhn) −W′ _(vhn)  (10)

The estimation value W_(vhn)′ in equation (10) can be obtained byequation (9). Accordingly, if equation (9) is substituted into W_(vhn)′in equation (10), the following equation can be found:

$\begin{matrix}{e_{vhn} = {W_{vhn} - {\sum\limits_{k = 0}^{9}{w_{{vhn},k}t_{k}}}}} & (11)\end{matrix}$where W_(vhn,k) represents the k-th term coefficient among thecoefficient seeds for generating the prediction coefficient W_(vhn). Inequation (11), t_(k) can be defined by the following equations.t₀=1t₁=vt₂=ht₃=v²t₄=vht₅=h²t₆=v³t₇=v²ht₈=vh²t₉=h³  (12)

The coefficient seed w_(vhn,k) that reduces the prediction error e_(vhn)in equation (10) or (11) to 0 or statistically minimizes the predictionerror e_(vhn) is the optimal coefficient seed for estimating theprediction coefficient. Generally, however, it is difficult to determinesuch a coefficient seed w_(vhn,k) for all prediction coefficients.

If, for example, the method of least squares, is employed as thestandard for representing that the coefficient seed w_(vhn,k) isoptimal, the optimal coefficient seed w_(vhn,k) can be obtained byminimizing the total error E of square errors expressed by the followingequation:

$\begin{matrix}{E = {\sum\limits_{v = 1}^{V}{\sum\limits_{h = 1}^{H}e_{vhn}^{2}}}} & (13)\end{matrix}$where V indicates the number of parameters v and H represents the numberof parameters h.

The minimum value of the total error E of the square errors in equation(13) can be given by the coefficient seed w_(vhn,k) that allows thevalue obtained by partially differentiating the total error E withrespect to the coefficient seed w_(vhn,k) to be 0, as expressed byequation (14).

$\begin{matrix}{\frac{\partial E}{\partial W_{{vhn},k}} = {{\sum\limits_{v = 1}^{V}\;{\sum\limits_{h = 1}^{H}\;{2( \frac{\partial e_{vhn}}{\partial W_{{vhn},k}} )e_{vhn}}}} = {{- {\sum\limits_{v = 1}^{V}\;{\sum\limits_{h = 1}^{H}\;{2t_{K}e_{vhn}}}}} = 0}}} & (14)\end{matrix}$

If X_(kl) and Y_(k) are defined by equations (15) and (16),respectively, equation (14) can be modified into normal equationsexpressed by equation (17).

$\begin{matrix}{X_{kl} = {\sum\limits_{v = 1}^{V}\;{\sum\limits_{h = 1}^{H}\;{t_{k}t_{l}}}}} & (15) \\{Y_{k} = {\sum\limits_{v = 1}^{V}\;{\sum\limits_{h = 1}^{H}\;{t_{k}W_{vhn}}}}} & (16) \\{{\begin{bmatrix}X_{00} & X_{01} & \cdots & X_{09} \\X_{10} & X_{11} & \cdots & X_{19} \\\vdots & \vdots & \ddots & \vdots \\X_{90} & X_{91} & \cdots & X_{99}\end{bmatrix}\begin{bmatrix}W_{n,0} \\W_{n,i} \\\vdots \\W_{n,9}\end{bmatrix}} = \begin{bmatrix}Y_{0} \\Y_{1} \\\vdots \\Y_{9}\end{bmatrix}} & (17)\end{matrix}$

The normal equations expressed by equation (17) can be solved withrespect to the coefficient seed w_(n,k) by using, for example, asweeping-out method (Gauss-Jordan elimination method).

By solving the normal equations in equation (17) for each class, theoptimal coefficient seed w_(n,k) that minimizes the total error E of theleast squares can be found for each class.

FIG. 7 is a block diagram illustrating the configuration of a learningdevice 601 that conducts learning for determining the coefficient seedw_(n,k) for each class by establishing and solving the normal equationsexpressed by equation (17).

The learning device 601 shown in FIG. 7 includes a band restrictionfilter 611, a class tap extracting unit 612, an ADRC processor 613, aprediction tap extracting unit 614, a normal equation generator 615, aprediction coefficient generator 616, a normal equation generator 617, acoefficient seed determining unit 618, and a coefficient seed memory619.

In the learning device 601, a plurality of supervisor images, whichserve as target natural images after performing prediction processing,read from a database (not shown) are input into the band restrictionfilter 611 and the normal equation generator 615. Parameters h and v arealso input from an external source to the band restriction filter 611and the normal equation generator 615 in response to an instruction froma user. In the learning device 601, every time one supervisor image isinput, all combinations of parameters h and v are input.

In response to the parameters h and v input from an external source, theband restriction filter 611 performs filtering processing forrestricting the bands of a supervisor image obtained from a database(not shown) in the vertical direction and in the horizontal direction.Accordingly, a learner image corresponding to a natural image beforeperforming prediction processing is generated for each combination ofparameters h and v. If the number of parameters h and the number ofparameters v are nine, the band restriction filter 611 generates 81learner images from one supervisor image in accordance with thecombinations of parameters h and v.

The band restriction filter 611 supplies the generated learner images tothe class tap extracting unit 612 and the prediction tap extracting unit614.

The configuration of the class tap extracting unit 612 is similar tothat of the class tap extracting unit 551 shown in FIG. 3.

The class tap extracting unit 612 sequentially selects the pixelsforming the supervisor image as subject supervisor pixels, and extractsclass taps having the same tap structure, such as that shown in FIG. 4,as that of the class taps extracted by the class tap extracting unit 551shown in FIG. 3. The class tap extracting unit 612 then supplies theclass taps to the ADRC processor 613.

The ADRC processor 613 performs ADRC processing on the pixel values ofthe pixels forming the class taps supplied from the class tap extractingunit 612, and sets the resulting ADRC code as the feature of the classtaps. The ADRC processor 613 determines the class based on the ADRC codeand supplies the determined class to the normal equation generator 615.

The configuration of the prediction tap extracting unit 614 is similarto that of the prediction tap extracting unit 556 shown in FIG. 3. Theprediction tap extracting unit 614 extracts, from a learner imagesupplied from the band restriction filter 611, as prediction taps, suchas those shown in FIG. 5, some of the pixels forming the learner imageused for predicting the pixel value of the subject supervisor pixel. Theprediction tap extracting unit 614 supplies the prediction taps to thenormal equation generator 615.

The normal equation generator 615 establishes the normal equationsexpressed by equation (8) for each class supplied from the ADRCprocessor 613 and for each combination of parameters h and v input froman external source by using the input supervisor image and a predictiontap supplied from the prediction tap extracting unit 614 as a learningpair used for learning the prediction coefficient W_(n). The normalequation generator 615 then supplies the normal equations to theprediction coefficient generator 616.

The prediction coefficient generator 616 solves the normal equations foreach class supplied from the normal equation generator 615 to determinethe prediction coefficient W_(n) that statistically minimizes theprediction error for each class and for each combination of parameters hand v. The prediction coefficient generator 616 then supplies theprediction coefficient W_(n) to the normal equation generator 617.

The normal equation generator 617 generates normal equations expressedby equation (17) for each class based on the prediction coefficientW_(vhn) supplied from the prediction coefficient generator 616, andoutputs the generated normal equations to the coefficient seeddetermining unit 618. The coefficient seed determining unit 618 solvesthe normal equations expressed by equation (17) for each class todetermine the coefficient seed w_(n,k) for each class, and stores thecoefficient seed w_(n,k) in the coefficient seed memory 619. Thecoefficient seed stored in the coefficient seed memory 619 is to bestored in the coefficient seed memory 553 shown in FIG. 3.

The positional relationship between the supervisor image and the learnerimages is described below with reference to FIG. 8.

In FIG. 8, the rhomboids represent the pixels of a supervisor image, andthe white circles represent the pixels of a learner image. In FIG. 8,the horizontal axis represents the horizontal position, while thevertical axis designates the vertical position.

The horizontal and vertical positions of the pixels of the supervisorimage are the same as those of the learner image. That is, thesupervisor image and the learner image are in phase with each other.

The learning processing performed by the learning device 601 shown inFIG. 7 is discussed below with reference to the flowchart in FIG. 9.

In step S601, in response to input parameters h and v, the bandrestriction filter 611 performs filtering processing for restricting thebands of an input supervisor image in the horizontal direction and inthe vertical direction to generate learner images. The band restrictionfilter 611 then supplies the generated learner images to the class tapextracting unit 612 and the prediction tap extracting unit 614.

In step S602, as in the class tap extracting portion 551 shown in FIG.3, the class tap extracting unit 612 selects one of the pixels formingthe supervisor image as a subject supervisor pixel.

In step S603, as in the class tap extracting portion 551 shown in FIG.3, the class tap extracting unit 612 extracts class taps, such as thoseshown in FIG. 4, from the learner image, and supplies the extractedclass taps to the ADRC processor 613.

In step S604, the ADRC processor 613 performs ADRC processing on thepixel values of the pixels forming the class taps. In step S605, theADRC processor 613 determines the class based on the ADRC code obtainedas a result of the ADRC processing, and supplies the determined class tothe normal equation generator 615.

In step S606, as in the prediction tap extracting portion 556 shown inFIG. 3, the prediction tap extracting unit 614 extracts prediction taps,such as those shown in FIG. 5, for the subject supervisor pixel from thelearner image supplied from the band restriction filter 611, andsupplies the prediction taps to the normal equation generator 615.

In step S607, the normal equation generator 615 extracts the subjectsupervisor pixel from the input supervisor image, and performs additionin equation (8) on the subject supervisor pixel and the learner imageforming the prediction taps for the subject supervisor pixel suppliedfrom the prediction tap extracting unit 614 for each combination ofparameters h and v and for each class supplied from the ADRC processor613.

In step S608, the class tap extracting unit 612 determines whether allthe pixels forming the input supervisor image have been selected assubject supervisor pixels. If it is determined in step S608 that not allthe pixels forming the supervisor image have been selected, the processproceeds to step S609. In step S609, the class tap extracting unit 612selects a pixel that has not been selected as a subject supervisorpixel. Then, the process returns to step S603, and step S603 and thesubsequent steps are repeated.

If it is determined in step S608 that all the pixels forming thesupervisor image have been selected as the subject supervisor pixels,the process proceeds to step S610. In step S610, the normal equationgenerator 615 supplies, as normal equations, a matrix on the left sideand the vector on the right side in equation (8) for each combination ofparameters h and v and for each class to the prediction coefficientgenerator 616.

In step S611, the prediction coefficient generator 616 solves the normalequations in equation (8) for each combination of parameters h and v andfor each class supplied from the normal equation generator 615 todetermine the prediction coefficient W_(vhn) for each combination ofparameters h and v and for each class. The prediction coefficientgenerator 616 outputs the determined prediction coefficient W_(vhn) tothe normal equation generator 617.

In step S612, the normal equation generator 617 generates normalequations in equation (17) for each class on the basis of the predictioncoefficient W_(vhn), and outputs the generated normal equations to thecoefficient seed determining unit 618.

In step S613, the coefficient seed determining unit 618 solves thenormal equations in equation (17) to determine the coefficient seedw_(n,k) for each class. In step S614, the coefficient seed w_(n,k) isstored in the coefficient seed memory 619. The coefficient seed w_(n,k)is to be stored in the coefficient seed memory 553 shown in FIG. 3.

As described above, the natural-image prediction unit 131 predicts ahigh-quality natural image by using the prediction coefficient w_(n)generated from the coefficient seed which is obtained by conductinglearning using a natural image. It is thus possible to enhance thequality of natural image components contained in an HD image suppliedfrom the output phase converter 112.

Additionally, the natural-image prediction unit 131 classifies subjectpixels in accordance with the feature of the waveforms of class taps.With this arrangement, the subject pixels of a natural image can beaccurately classified. The natural-image prediction unit 131 can predicta high-quality natural image from the HD image by using a predictioncoefficient generated from a coefficient seed obtained by conductinglearning for each class. Accordingly, the natural-image prediction unit131 can suppress noise contained in the natural image with higherprecision and thus outputs a higher-quality natural image.

FIG. 10 is a block diagram illustrating the configuration of theartificial-image prediction unit 132 shown in FIG. 1.

The artificial-image prediction unit 132 includes a classificationportion 651, a coefficient seed memory 652, a prediction coefficientgenerator 653, and a prediction coefficient memory 654, and a predictionportion 655. The artificial-image prediction unit 132 predicts ahigh-quality artificial image after eliminating noise from artificialimage components contained in a progressive HD image supplied from theoutput phase converter 112.

The HD image supplied from the output phase converter 112 is input intothe classification portion 651 and the prediction portion 655. Theclassification portion 651 sequentially selects the pixels forming thehigh-quality artificial image determined from the HD image as subjectpixels, and allocates the subject pixels into some classes in accordancewith the feature of the phase of the HD image. The classificationportion 651 then supplies the classes to the prediction coefficientmemory 654.

The coefficient seed memory 652 is formed of, for example, a read onlymemory (ROM), and stores a coefficient seed obtained by conductinglearning, which is discussed below with reference to FIGS. 17 through19, for each class.

The prediction coefficient generator 653 generates the predictioncoefficient W_(n) from the coefficient seed w_(n,k) read from thecoefficient seed memory 652 by using the polynomial expressed byequation (9) containing the parameters h and v input from the user, andstores the generated prediction coefficient W_(n) in the predictioncoefficient memory 654.

The prediction coefficient memory 654 reads out the predictioncoefficient W_(n) based on the class supplied from the classificationportion 651, and supplies the prediction coefficient W_(n) for thatclass to the prediction portion 655.

The prediction portion 655 performs predetermined prediction computationfor determining the prediction value of the true value of the subjectpixel by using the HD image and the prediction coefficient W_(n)supplied from the prediction coefficient memory 654. Accordingly, theprediction portion 655 predicts the pixel value of the subject pixel,i.e., the pixel value of the subject pixel forming the high-qualityartificial image, and outputs the predicted pixel value to thesynthesizer 133 shown in FIG. 1.

FIG. 11 is a block diagram illustrating the detailed configuration ofthe classification portion 651.

The classification portion 651 shown in FIG. 11 includes a class tapextracting portion 671, a difference calculator 672, and an ADRCprocessor 673.

The class tap extracting portion 671 extracts, as class taps, some ofthe pixels forming the HD image for classifying the subject pixel, andsupplies the class taps to the difference calculator 672.

Among the pixels forming the class taps supplied from the class tapextracting portion 671, the difference calculator 672 calculates, as thefeature of the phase of the class taps, the absolute value of thedifference of the pixel values of two adjacent pixels (hereinaftersimply referred to as “adjacent pixels”) for each set of adjacentpixels. Such an absolute value is hereinafter referred to as the“adjacent difference absolute value”. The difference calculator 672supplies the adjacent difference absolute value of each set of adjacentpixels to the ADRC processor 673.

The ADRC processor 673 performs one-bit ADRC processing on the adjacentdifference absolute values supplied from the difference calculator 672.More specifically, the ADRC processor 673 divides the adjacentdifference absolute values of the class taps by the average of themaximum value MAX and the minimum value MIN to re-quantize each adjacentdifferent absolute value into one bit with the decimal fractionsomitted. That is, the ADRC processor 673 binarizes the adjacentdifference absolute values.

The ADRC processor 673 arranges the one-bit pixel values in apredetermined order, resulting in a bit string, which is determined asthe class of the subject pixel. Accordingly, the class serves as phaseinformation concerning the positions of edges in the class taps. Thatis, the class represents the value degenerated from the phase of theclass taps. The ADRC processor 673 supplies the determined class to theprediction coefficient memory 654 shown in FIG. 10.

In this manner, the classification portion 651 classifies the subjectpixel in accordance with the feature of the phase of the class tapsobtained from the adjacent difference absolute value of each set ofadjacent pixels.

FIG. 12 illustrates an example of the tap structure of the class tapsextracted by the class tap extracting portion 671 shown in FIG. 11.However, this is an example only, and the tap structure of the classtaps may be different from that shown in FIG. 12.

In FIG. 12, among an HD image supplied from the output phase converter112 shown in FIG. 1, nine pixels including a pixel p124 corresponding toa subject pixel and two pixels adjacent to the pixel p124 in each of theupward, leftward, rightward, and downward directions, i.e., pixels p120,p121, p122, p123, p125, p126, p127, and p128, are disposed. That is, aso-called “cross-shaped class tap” structure is formed.

The difference calculator 672 shown in FIG. 11 calculates eight adjacentdifference absolute values d0 through d7 between pixels p120 and p121,pixels p121 and p124, pixels p122 and p123, pixels p123 and p124, pixelsp124 and p125, pixels p125 and p126, p124 and p127, and p127 and p128,respectively, and supplies the calculated adjacent difference absolutevalues d0 through d7 to the ADRC processor 673. As a result, the 8-bitclass is output from the ADRC processor 673.

FIG. 13 is a block diagram illustrating the detailed configuration ofthe prediction portion 655 shown in FIG. 10.

The prediction portion 655 shown in FIG. 13 includes a prediction tapextracting portion 691 and a prediction computation portion 692.

The prediction tap extracting portion 691 extracts, as prediction taps,some of the pixels forming the HD image used for predicting the pixelvalue of a subject pixel.

More specifically, the prediction tap extracting portion 691 extracts,from the HD image, as prediction taps, pixels corresponding to thesubject pixel, for example, a plurality of pixels of the HD imagespatially closer to the subject pixel. The prediction tap extractingportion 691 supplies the extracted prediction taps to the predictioncomputation portion 692.

The prediction taps and the class taps may have the same tap structureor different tap structures.

The prediction computation portion 692 receives, not only the predictiontaps from the prediction tap extracting portion 691, but also theprediction coefficient from the prediction coefficient memory 654 shownin FIG. 10. The prediction computation portion 692 performs predictioncomputation expressed by equation (1) to determine the prediction valueof the true value of the subject pixel by using the prediction taps andthe prediction coefficient. Accordingly, the prediction computationportion 692 predicts pixel value of the subject pixel, i.e., the pixelvalue of the subject pixel forming the high-quality artificial image,and outputs the predicted pixel value to the synthesizer 133 shown inFIG. 1.

FIG. 14 illustrates an example of the tap structure of the predictiontaps extracted by the prediction tap extracting portion 691 shown inFIG. 13. However, this is an example only, and the tap structure of theprediction taps may be different from that shown in FIG. 14.

In FIG. 14, the prediction taps are formed of 13 pixels. Morespecifically, in FIG. 14, among the HD image supplied from the outputphase converter 112, five pixels p140, p142, p146, p150, and p152vertically arranged around pixel p146 corresponding to the subjectpixel, three pixels p141, p145, and p149 vertically arranged aroundp145, which is left-adjacent to pixel p146, three pixels p143, p147, andp151 vertically arranged around p147, which is right-adjacent to pixelp146, and two pixels p144 and p148 away from pixel p146 in the left andright directions by two pixels, are disposed. That is, a generallyrhomboid prediction tap structure is formed.

Details of the artificial-image prediction processing in step S5 in FIG.2 performed by the artificial-image prediction unit 132 shown in FIG. 10are discussed below.

In step S701, the classification portion 651 performs classificationprocessing for classifying a predetermined subject pixel of the pixelsforming a high-quality artificial image in accordance with the featureof the phase of the HD image corresponding to the subject pixel. Detailsof the classification processing are discussed below with reference toFIG. 16.

In step S702, the coefficient seed memory 652 reads out the coefficientseed w_(n,k) and outputs it to the prediction coefficient generator 653.In step S703, the prediction coefficient generator 653 generates theprediction coefficient W_(n) from the coefficient seed w_(n,k) on thebasis of the parameters h and v input from the user by using thepolynomial expressed by equation (9) containing the parameters h and v,and supplies the generated prediction coefficient W_(n) to theprediction coefficient memory 654.

In step S704, the prediction coefficient memory 654 reads out theprediction coefficient W_(n) based on the class determined by theclassification portion 651, and supplies the read prediction coefficientW_(n) to the prediction computation portion 692 of the predictionportion 655.

In step S705, the prediction tap extracting portion 691 extracts, fromthe HD image supplied from the output phase converter 112, as predictiontaps, such as those shown in FIG. 14, some of the pixels forming the HDimage used for predicting the pixel value of the subject pixel, andsupplies the extracted prediction taps to the prediction computationportion 692.

In step S706, the prediction computation portion 692 performs predictioncomputation expressed by equation (1) by using the prediction tapssupplied from the prediction tap extracting portion 691 and theprediction coefficient W_(n) supplied from the prediction coefficientmemory 654 to determine the pixel value of the subject pixel forming thehigh-quality artificial image. In step S707, the prediction computationportion 692 outputs the pixel value of the subject pixel forming thehigh-quality artificial image determined in step S706 to the synthesizer133 shown in FIG. 1.

In step S708, the classification portion 651 determines whether all thepixels forming the high-quality artificial image have been selected asthe subject pixels. If it is determined in step S708 that not all thepixels have been selected, the process proceeds to step S709. In stepS709, the classification portion 651 determines a pixel that has notbeen selected as the next subject pixel and then returns to step S701.Steps S701 and the subsequent steps are repeated.

If the classification portion 651 determines in step S708 that all thepixels forming the high-quality artificial image have been selected asthe subject pixels, the artificial-image prediction processing iscompleted.

As discussed above, the artificial-image prediction unit 132 predicts ahigh-quality artificial image from an HD image supplied from the outputphase converter 112, and outputs the predicted image. That is, theartificial-image prediction unit 132 converts an HD image into ahigh-quality artificial image and outputs it.

Details of the classification processing in step S701 in FIG. 15 arediscussed below with reference to the flowchart in FIG. 16.

In step S721, the class tap extracting portion 671 shown in FIG. 11 ofthe classification portion 651 extracts, as class taps, such as thoseshown in FIG. 12, some of the pixels forming the HD image used forclassifying the subject pixel, and supplies the extracted class taps tothe difference calculator 672.

In step S722, the difference calculator 672 calculates, among the pixelsforming the class taps supplied from the class tap extracting portion671, the adjacent difference absolute value of each set of adjacentpixels, and supplies the calculated adjacent difference absolute valuesto the ADRC processor 673.

In step S723, the ADRC processor 673 performs one-bit ADRC processing onthe adjacent difference absolute values supplied from the differencecalculator 672. The ADRC processor 673 then determines the resultingADRC bit string as the class to classify the subject pixel. The ADRCprocessor 673 then supplies the determined class to the predictioncoefficient memory 654 shown in FIG. 10. The process then returns tostep S701 in FIG. 15.

FIG. 17 is a block diagram illustrating the configuration of a learningdevice 811 that conducts learning for determining coefficient seeds tobe stored in the coefficient memory 652 shown in FIG. 10.

The learning device 811 shown in FIG. 17 includes a learner imagegenerator 821, a classification unit 822, a generator 823, a coefficientgenerator 824, a normal equation generator 825, a coefficient seeddetermining unit 826, and a coefficient seed memory 827.

The learning device 811 conducts learning for coefficient seeds in amanner similar to learning conducted for the coefficient seed w_(n,k)stored in the coefficient seed memory 553 (FIG. 3) of the natural-imageprediction unit 131. More specifically, by using, as a supervisor image,an HD image corresponding to a target artificial image after performingprediction processing, and by using, as a learner image, an HD imagecorresponding to an artificial image before performing predictionprocessing, the learning device 811 solves the normal equationsexpressed by equation (8) for each class and for each combination ofexternally input parameters h and v which are externally input inresponse to an instruction from the user. As a result, the learningdevice 811 can determine the prediction coefficient W_(vhn), which isthe prediction coefficient W_(n) for each class and for each combinationof parameters h and v.

Then, the learning device 811 solves the normal equations expressed byequation (17) generated for each class based on the predictioncoefficient W_(vhn), thereby generating the coefficient seed w_(n,k) foreach class. The learning device 811 then stores the coefficient seedw_(n,k).

A plurality of supervisor images read from a database (not shown) areinput into the learning device 811 and are supplied to the learner imagegenerator 821 and the generator 823. Parameters h and v are also inputinto the learning device 811 and are supplied to the learner imagegenerator 821 and the generator 823.

The learner image generator 821 is formed of, for example, a low-passfilter. The learner image generator 821 decreases the quality of asupervisor image, which is an artificial image, obtained from a database(not shown) in accordance with the parameters h and v, therebygenerating a learner image for each combination of parameters h and v.The learner image generator 821 supplies the generated learner image tothe classification unit 822 and the generator 823.

The configuration of the classification unit 822 is similar to that ofthe classification portion 651 shown in FIG. 11 of the artificial-imageprediction unit 132. The classification unit 822 sequentially selectsthe pixels forming the supervisor image as the subject supervisorpixels, and extracts, from the learner image, class taps having the sametap structure as the class taps (FIG. 12) extracted by the class tapextracting portion 671 shown in FIG. 11 for each subject supervisorpixel.

The classification unit 822 calculates the adjacent difference absolutevalue of each set of adjacent pixels among the pixels forming the classtaps, and performs one-bit ADRC processing on the adjacent differenceabsolute values. The classification unit 822 determines the resultingbit string as the class of the subject supervisor pixel, and suppliesthe determined class to the generator 823.

The generator 823 establishes the normal equations expressed by equation(8) for each combination of externally input parameters h and v and foreach class supplied from the classification unit 822 by using learningpairs, and supplies the normal equations to the coefficient generator824. The learning pairs are formed of a supervisor image and the learnerimages supplied from the learner image generator 821 and are used forlearning prediction coefficients.

The coefficient generator 824 solves the normal equations supplied fromthe generator 823 for each combination of parameters h and v and foreach class to determine the prediction coefficient W_(vhn) for eachcombination of parameters h and v and for each class, and outputs theprediction coefficient W_(vhn) to the normal equation generator 825.

The normal equation generator 825 generates normal equations expressedby equation (17) for each class based on the prediction coefficientW_(vhn), and outputs the generated normal equations to the coefficientseed determining unit 826. The coefficient seed determining unit 826solves the normal equations to determine the coefficient seed w_(n,k)and stores it in the coefficient seed memory 827. The coefficient seedw_(n,k) stored in the coefficient seed memory 827 is to be stored in thecoefficient seed memory 652 shown in FIG. 10.

FIG. 18 is a block diagram illustrating the detailed configuration ofthe generator 823 shown in FIG. 17.

The generator 823 shown in FIG. 18 includes a prediction tap extractingportion 831 and a normal equation generating portion 832.

The learner image and the supervisor image of a learning pair input intothe generator 823 are supplied to the prediction tap extracting portion831 and the normal equation generating portion 832, respectively.

The prediction tap extracting portion 831 sequentially selects thepixels forming the supervisor image of the learning pair as the subjectsupervisor pixels. The prediction tap extracting portion 831 thenextracts, from the learner image of the learning pair, prediction tapshaving the same tap structure as the prediction taps (FIG. 14) extractedby the prediction tap extracting portion 691 shown in FIG. 13, andsupplies the prediction taps to the normal equation generating portion832.

The normal equation generating portion 832 extracts the subjectsupervisor pixel from the supervisor image, and performs additionprocessing on the subject supervisor pixel and the learner image formingthe prediction taps extracted for the subject supervisor pixel for eachcombination of externally input parameters h and v and for each classsupplied from the classification unit 822.

The normal equation generating portion 832 performs the above-describedaddition processing by setting all the pixels forming the supervisorimage input into the learning device 811 as the subject supervisorpixels to establish the normal equations expressed by equation (8) foreach class, and then supplies the normal equations to the coefficientgenerator 824 shown in FIG. 17.

The learning processing performed by the learning device 811 shown inFIG. 17 is described below with reference to the flowchart in FIG. 19.

In step S741, the learner image generator 821 generates learner imagesfrom an input supervisor image in accordance with externally inputparameters h and v, and supplies the generated learner images to theclassification unit 822 and the generator 823.

In step S742, the classification unit 822 performs classificationprocessing, as in the classification processing shown in FIG. 16, forclassifying a predetermined subject supervisor pixel of the supervisorimage in accordance with the phase of the learner image associated withthe subject supervisor pixel. The classification unit 822 supplies thedetermined class to the normal equation generating portion 832 (FIG. 18)of the generator 823.

In step S743, the prediction tap extracting portion 831 shown in FIG. 18extracts prediction taps for the subject supervisor pixel from thelearner image supplied from the learner image generator 821, andsupplies the extracted prediction taps to the normal equation generatingportion 832.

In step S744, the normal equation generating portion 832 extracts thesubject supervisor pixel from the input supervisor image, and performsaddition processing expressed by equation (8) on the subject supervisorpixel and the learner image forming the associated prediction tapssupplied from the prediction tap extracting portion 831 for eachcombination of parameters h and v and for each class supplied from theclassification unit 822.

In step S745, the classification unit 822 determines whether all thepixels forming the supervisor image have been selected as the subjectsupervisor pixels. If it is determined in step S745 that not all thepixels have been selected as the subject supervisor pixels, the processproceeds to step S746. In step S746, the classification unit 822 selectsa pixel of the supervisor image which has not been selected as the nextsubject supervisor pixel, and then returns to step S742. Steps S742 andthe subsequent steps are repeated.

If it is determined in step S745 that all the pixels forming thesupervisor image have been selected as the subject supervisor pixels,the process proceeds to step S747. In step S747, the normal equationgenerating portion 832 supplies the matrix on the left side and thevector on the right side in equation (8) for each combination ofparameters h and v and for each class to the coefficient generator 824as the normal equations.

In step S748, the coefficient generator 824 solves the normal equationscontaining the matrix on the left side and the vector on the right sidein equation (8) for each combination of parameters h and v and for eachclass to determine the prediction coefficient W_(vhn) for eachcombination of parameters h and v and for each class. The coefficientgenerator 824 then supplies the determined prediction coefficientW_(vhn) to the normal equation generator 825.

In step S749, the normal equation generator 825 generates normalequations expressed by equation (17) for each class based on theprediction coefficient W_(vhn), and outputs the normal equations to thecoefficient seed determining unit 826. In step S750, the coefficientseed determining unit 826 solves the normal equations expressed byequation (17) for each class to determine the coefficient seed w_(n,k)for each class. In step S751, the coefficient seed w_(n,k) is stored inthe coefficient seed memory 827 and is also stored in the coefficientseed memory 652 shown in FIG. 10.

As discussed above, the artificial-image prediction unit 132 predicts ahigh-quality artificial image by using a prediction coefficientgenerated from a coefficient seed obtained by conducting learning usingan artificial image. It is thus possible to enhance the quality of theartificial image components contained in the HD image supplied from theoutput phase converter 112.

Additionally, the artificial-image prediction unit 132 classifiessubject pixels in accordance with the positions of the edges of classtaps as the feature of the phase. With this arrangement, the subjectpixels of an artificial image having small number of grayscale levelsand distinct phase information can be accurately classified.Accordingly, the artificial-image prediction unit 132 can predict ahigh-quality artificial image from an HD image by using a predictioncoefficient generated from a coefficient seed obtained by conductinglearning for each class. As a result, it is possible to suppress noisecontained in the artificial image with higher precision and to output ahigher-quality artificial image.

A description is now given, with reference to FIG. 20, of the detailedconfiguration of the natural-image/artificial-image determining unit114.

The natural-image/artificial-image determining unit 114 includes abroad-range feature extracting portion 911, a broad-rangedegree-of-artificiality calculator 912, a degree-of-artificialitygenerator 913, a narrow-range feature extracting portion 914, and anarrow-range degree-of-artificiality calculator 915.

The broad-range feature extracting portion 911 includes a broad edgeparameter (BEP) extracting portion 931, a broad flat parameter (BFP)extracting portion 932, and a feature synthesizer 933. The broad-rangefeature extracting portion 911 extracts a broad-range feature from eachpixel forming an HD image supplied from the output phase converter 112,and supplies the extracted broad-range feature to the broad-rangedegree-of-artificiality calculator 912.

The BEP extracting portion 931 obtains, for each pixel of the HD imagesupplied from the output phase converter 112, a value based on thedynamic range of the pixel values of pixels contained in a referencearea set for the subject pixel. The BEP extracting portion 931 then setsthe obtained value as the broad-range feature BEP and supplies thebroad-range feature BEP to the BFP extracting portion 932 and thefeature synthesizer 933. The detailed configuration of the BEPextracting portion 931 is given below with reference to FIG. 21. Thepixels contained in the reference area and the broad-range feature BEPare also referred to as the “reference pixels” and the “BEP”,respectively. Also, in the following description, the subject pixel is apixel forming the HD image supplied from the output phase converter 112and is to be processed by the natural-image/artificial-image determiningunit 114.

The BFP extracting portion 932 obtains, for each pixel forming the HDimage supplied from the output phase converter 112, a value based on theBEP and the difference absolute values of adjacent pixels among thereference pixels set for the subject pixel. The BFP extracting portion932 then sets the obtained value as the broad-range feature BFP andsupplies it to the feature synthesizer 933. The detailed configurationof the BFP extracting portion 932 is given below with reference to FIG.22. The broad-range feature BFP is also simply referred to as the “BFP”.

The feature synthesizer 933 combines the BEP supplied from the BEPextracting portion 931 and the BFP supplied from the BFP extractingportion 932 to generate a broad-range feature, and supplies it to thebroad-range degree-of-artificiality calculator 912. That is, thebroad-range feature is formed of two types of features, such as theabove-described BEP and BFP, and indicates the feature of an imagelocated in a broad range that contains the subject pixel.

The broad-range degree-of-artificiality calculator 912 includes afeature separator 951, a broad-range boundary comparator 952, abroad-range boundary memory 953, and a dividing point calculator 954.The broad-range degree-of-artificiality calculator 912 calculates thebroad-range degree of artificiality Art_(b) on the basis of thebroad-range feature supplied from the broad-range feature extractingportion 911, and supplies the calculated broad-range degree ofartificiality Art_(b) to the degree-of-artificiality generator 913.

The feature separator 951 obtains the broad-range feature supplied fromthe broad-range feature extracting portion 911, and supplies theobtained feature to the broad-range boundary comparator 952. That is, inthis example, since the broad-range feature includes the BEP and BFP,the feature separator 951 separates the BEP and BFP and supplies them tothe broad-range boundary comparator 952.

The broad-range boundary memory 953 stores broad-range boundaries, whichhave been statistically determined from a plurality of artificial imagesand natural images, indicating whether image components contained inartificial images and natural images belong to an artificial image or anatural image. The broad-range boundary comparator 952 compares the twotypes of features, i.e., the BEP and BFP, supplied from the featureseparator 951 with the broad-range boundaries, and supplies comparisonresults to the dividing point calculator 954.

The dividing point calculator 954 calculates the broad-range degree ofartificiality Art_(b) corresponding to the broad-range feature on thebasis of the comparison results supplied from the broad-range boundarycomparator 952, and supplies the calculated broad-range degree ofartificiality Art_(b) to the degree-of-artificiality generator 913.

The narrow-range feature extracting portion 914 includes a primarynarrow discrimination parameter (PNDP) extracting portion 971, asecondary narrow discrimination parameter (SNDP) extracting portion 972,and a feature synthesizer 973. The narrow-range feature extractingportion 914 extracts a narrow-range feature from each pixel of the HDimage supplied from the output phase converter 112, and supplies theextracted narrow-range feature to the narrow-rangedegree-of-artificiality calculator 915.

The PNDP extracting portion 971 obtains, for each pixel of the HD imagesupplied from the output phase converter 112, the value based on thedynamic range of the pixel values of the pixels of a long tap areaincluded in a reference area which is set for the subject pixel. ThePNDP extracting portion 971 then sets the obtained value as the featurePNDP forming the narrow-range feature, and supplies the feature PNDP tothe SNDP extracting portion 972 and the feature synthesizer 973. Thedetailed configuration of the PNDP extracting portion 971 is discussedbelow with reference to FIG. 23. The feature PNDP is also simplyreferred to as the “PNDP”.

The SNDP extracting portion 972 extracts, as the narrow-range feature,for each pixel of the HD image supplied from the output phase converter112, the feature SNDP obtained based on the PNDP and the dynamic rangeof the pixel values of the pixels of a short tap area contained in along tap area included in a reference area which is set for the subjectpixel. The SNDP extracting portion 972 then supplies the feature SNDP tothe feature synthesizer 973 as the narrow-range feature. The detailedconfiguration of the SNDP extracting portion 972 is discussed below withreference to FIG. 24. The feature SNDP is also simply referred to as the“SNDP”.

The feature synthesizer 973 combines the PNDP supplied from the PNDPextracting portion 971 with the SNDP supplied from the SNDP extractingportion 972 to generate a narrow-range feature, and supplies thenarrow-range feature to the narrow-range degree-of-artificialitycalculator 915. That is, the narrow-range feature is formed of two typesof features, i.e., the PDNP and SNDP, and indicates the feature of animage located in a narrow range that contains the subject pixel.

The narrow-range degree-of-artificiality calculator 915 includes afeature separator 991, a narrow-range boundary comparator 992, anarrow-range boundary memory 993, and a dividing point calculator 994.The narrow-range degree-of-artificiality calculator 915 calculates thenarrow-range degree of artificiality Art_(n) on the basis of thenarrow-range feature supplied from the narrow-range feature extractingportion 914, and supplies the calculated narrow-range degree ofartificiality Art_(n) to the degree-of-artificiality generator 913.

The feature separator 991 separates the narrow-range feature suppliedfrom the narrow-range feature extracting portion 914, and supplies theseparated feature to the narrow-range boundary comparator 992. That is,in this example, since the narrow-range feature includes the PNDP andSNDP, the feature separator 991 separates the PNDP and SNDP and suppliesthem to the narrow-range boundary comparator 992.

The narrow-range boundary memory 993 stores narrow-range boundaries,which have been statistically determined from a plurality of artificialimages and natural images, indicating whether image components containedin artificial images and natural images belong to an artificial image ora natural image. The narrow-range boundary comparator 992 compares thetwo types of features, i.e., the PNDP and SNDP, supplied from thefeature separator 991 with the narrow-range boundaries, and suppliescomparison results to the dividing point calculator 994.

The dividing point calculator 994 calculates the narrow-range degree ofartificiality Art_(n) corresponding to the narrow-range feature on thebasis of the comparison results supplied from the narrow-range boundarycomparator 992, and supplies the calculated narrow-range degree ofartificiality Art_(n) to the degree-of-artificiality generator 913.

The degree-of-artificiality generator 913 combines the broad-rangedegree of artificiality Art_(b) supplied from the broad-rangedegree-of-artificiality calculator 912 with the narrow-range degree ofartificiality Art_(n) supplied from the narrow-rangedegree-of-artificiality calculator 915 to generate the degree ofartificiality Art, and outputs it to the image processor 113.

The detailed configuration of the BEP extracting portion 931 isdiscussed below with reference to FIG. 21.

The BEP extracting portion 931 shown in FIG. 21 includes a buffer 1011,a reference-pixel extracting portion 1012, a weight calculator 1013, aninter-pixel difference calculator 1014, a multiplier 1015, a storageportion 1016, a maximum/minimum-value extracting portion 1017, and a BEPcalculator 1018.

The buffer 1011 temporarily stores an HD image supplied from the outputphase converter 112, and supplies the HD image to the reference-pixelextracting portion 1012 if necessary. The reference-pixel extractingportion 1012 sequentially reads out reference pixels for each subjectpixel, and supplies the read reference pixels to the weight calculator1013 and the inter-pixel difference calculator 1014. The referencepixels are pixels within a predetermined area which is set for eachsubject pixel, e.g., all the pixels contained in an n-pixel×n-pixel areaaround the subject pixel. The number and the arrangement of referencepixels are arbitrary as long as the reference pixels are set for asubject pixel.

The weight calculator 1013 calculates a weight in accordance with thedistance between the subject pixel and each reference pixel suppliedfrom the reference-pixel extracting portion 1012, and supplies thecalculated weight to the multiplier 1015. The inter-pixel differencecalculator 1014 determines the difference between the subject pixel andeach reference pixel supplied from the reference-pixel extractingportion 1012, and supplies the calculated difference to the multiplier1015. The multiplier 1015 multiplies the difference calculated by theinter-pixel difference calculator 1014 by the weight calculated by theweight calculator 1013, and stores the resulting value in the storageportion 1016.

When all the values obtained by multiplying the differences between thesubject pixel and all the reference pixels by the weights are stored inthe storage portion 1016, the maximum/minimum-value extracting portion1017 extracts the maximum value and the minimum value, and supplies themaximum value and the minimum value to the BEP calculator 1018. The BEPcalculator 1018 outputs, as the BEP of the subject pixel, the valueobtained by multiplying the difference between the subject pixel andeach reference pixel, i.e., the dynamic range obtained from thedifferences between the subject pixel and the reference pixels, by theweight calculated based on the distance between the subject pixel andthe corresponding reference pixel. The value obtained by multiplying thedifference dynamic range by the weight is also referred to as the“dynamic range based on the distance from the subject pixel” or simplyreferred to as the “difference dynamic range”.

The detailed configuration of the BFP extracting portion 932 is nowdiscussed below with reference to FIG. 22.

The BFP extracting portion 932 includes a buffer 1031, a reference-pixelextracting portion 1032, an adjacent-pixel difference calculator 1033, afunction transformer 1034, a weight calculator 1035, a multiplier 1036,a storage portion 1037, and a BFP calculator 1038.

The buffer 1031 temporarily stores HD images supplied from the outputphase converter 112, and sequentially supplies the HD images to thereference-pixel extracting portion 1032 if necessary. Thereference-pixel extracting portion 1032 sequentially reads out thereference pixels for each subject pixel, and supplies the referencepixels to the adjacent-pixel difference calculator 1033 and the weightcalculator 1035.

The adjacent-pixel difference calculator 1033 calculates the absolutevalues of the differences between adjacent pixels of all the referencepixels supplied from the reference-pixel extracting portion 1032, andsupplies the calculated absolute values to the function transformer1034. The function transformer 1034 sets a transform function based onthe BEP supplied from the BEP extracting portion 931, and thentransforms the adjacent-pixel difference absolute values supplied fromthe adjacent-pixel difference calculator 1033 by the transform function,and supplies the transformed absolute values to the multiplier 1036.

The weight calculator 1035 calculates, for the position of eachreference pixel supplied from the reference-pixel extracting portion1032, the weight based on the distance from the subject pixel, andsupplies the calculated weight to the multiplier 1036. The multiplier1036 multiplies the adjacent-pixel difference absolute valuestransformed by the function and supplied from the function transformer1034 by the weights supplied from the weight calculator 1035, and storesthe multiplied values in the storage portion 1037.

The BFP calculator 1038 cumulatively adds the multiplied values storedin the storage portion 1037, and outputs the added value as the BFP ofthe subject pixel.

The detailed configuration of the PNDP extracting portion 971 isdiscussed below with reference to FIG. 23.

The PNDP extracting portion 971 includes a buffer 1051, areference-pixel extracting portion 1052, a long-tap extracting portion1053, a pixel-value storage portion 1054, a maximum/minimum-valueextracting portion 1055, and a PNDP calculator 1056.

The buffer 1051 temporarily stores HD images supplied from the outputphase converter 112, and sequentially supplies the HD images to thereference-pixel extracting portion 1052 if necessary. Thereference-pixel extracting portion 1052 sequentially reads out thereference pixels for each subject pixel, and supplies the read referencepixels to the long-tap extracting portion 1053.

The long-tap extracting portion 1053 extracts pixels from an areacontained in the reference area, i.e., from an area smaller than orequal to the reference pixel area, and supplies the extracted pixels tothe pixel-value storage portion 1054. The long-tap extracting portion1053 also supplies long tap information to a short-tap extractingportion 1073 of the SNDP extracting portion 972. When all the pixelsforming the long tap have been extracted, the long-tap extractingportion 1053 supplies information that all the pixels forming the longtap have been extracted to the maximum/minimum-value extracting portion1055.

The maximum/minimum-value extracting portion 1055 extracts the maximumvalue and the minimum value of all the pixel values of the long tapstored in the pixel-value storage portion 1054, and supplies theextracted maximum and minimum values to the PNDP calculator 1056.

The PNDP calculator 1056 determines the reference-pixel dynamic range ofthe pixel values from the difference between the maximum value and theminimum value supplied from the maximum/minimum-value extracting portion1055 (such a dynamic range is simply referred to as a “pixel-valuedynamic range”), and outputs the pixel-value dynamic range as the PNDPof the subject pixel. The pixel-value dynamic range is the valueobtained by subtracting the minimum value from the maximum value of allthe pixel values contained n a predetermined area.

The detailed configuration of the SNDP extracting portion 972 isdiscussed below with reference to FIG. 24.

The SNDP extracting portion 972 includes a buffer 1071, areference-pixel extracting portion 1072, the short-tap extractingportion 1073, a pixel value storage portion 1074, amaximum/minimum-value extracting portion 1075, a dynamic range (DR)calculator 1076, a DR storage portion 1077, and an SNDP selector 1078.

The buffer 1071 temporarily stores HD images supplied from the outputphase converter 112, and sequentially supplies the HD images to thereference-pixel extracting portion 1072 if necessary. Thereference-pixel extracting portion 1072 sequentially reads out thereference pixels for each subject pixel, and supplies the read referencepixels to the short-tap extracting portion 1073.

Based on information concerning the long tap supplied from the long-tapextracting portion 1053, the short-tap extracting portion 1073 extractspixels from a plurality of areas containing the subject pixel, which aresmaller than or equal to the long tap area, and supplies the extractedpixels to the pixel value storage portion 1074. That is, for one longtap, a plurality of patterns may be set for short taps. When all thepatterns for a plurality of short taps have been extracted, theshort-tap extracting portion 1073 supplies information that all thepatterns have been extracted to the SNDP selector 1078.

The maximum/minimum-value extracting portion 1075 extracts the maximumvalue and the minimum value of all the pixel values contained in eachshort tap and stored in the pixel value storage portion 1074, andsupplies the extracted maximum and minimum values to the DR calculator1076.

The DR calculator 1076 determines the pixel-value dynamic range from thedifference between the maximum and minimum values supplied from themaximum/minimum-value extracting portion 1075, and stores thepixel-value dynamic range in the DR storage portion 1077.

Upon receiving the information that all the patterns have been extractedfor short taps from the short-tap extracting portion 1073, the SNDPselector 1078 selects the minimum dynamic range from the pixel-valuedynamic ranges determined for all the patterns of the short taps andstored in the DR storage portion 1077, and outputs the selected minimumdynamic range as the SNDP.

The natural-image/artificial-image determination processing is discussedbelow with reference to the flowchart in FIG. 25.

In step S831, the BEP extracting portion 931 of the broad-range featureextracting portion 911 performs BEP extraction processing to extract theBEP for each pixel from an HD image supplied from the output phaseconverter 112, and supplies the extracted BEP to the feature synthesizer933.

A description is now given, with reference to the flowchart in FIG. 26,of the BEP extraction processing performed by the BEP extracting portion931 shown in FIG. 21.

In step S851, the buffer 1011 temporarily stores HD images supplied fromthe output phase converter 112.

In step S852, the reference-pixel extracting portion 1012 selects anunprocessed subject pixel received from the buffer 1011 as a subjectpixel. In step S853, the reference-pixel extracting portion 1012 readsout reference pixels which are set for the subject pixel and suppliesthe read reference pixels to the weight calculator 1013 and theinter-pixel difference calculator 1014. The reference pixels are pixelsin a predetermined area which is set for the subject pixel, for example,all the pixels contained in an n-pixel×n-pixel area around the subjectpixel, as shown in FIG. 27.

In FIG. 27, in a reference area R of an image, reference pixels Q(i,j)are indicated by the white circles on the two-dimensional coordinates inthe x direction and in the y direction around the subject pixel Q(0,0)indicated by the hatched portion. The pixels contained in the areasurrounded by the broken lines (n×n) around the subject pixel Q(0,0) arereference pixels Q(i,j) that can be represented by −(n−1)/2≦i≦(n−1)/2,−(n−1)/2≦j≦(n−1)/2, where i and j are integers.

In step S854, the inter-pixel difference calculator 1014 calculates thedifference between the subject pixel and an unprocessed reference pixel,and supplies the calculated difference to the multiplier 1015. That is,in FIG. 27, the inter-pixel difference calculator 1014 calculates(Q(0,0)−Q(i,j)) as the inter-pixel difference, and supplies thecalculated difference to the multiplier 1015. Q(0,0) and Q(i,j) in theinter-pixel difference represent the pixel values of the subject pixelQ(0,0) and the reference pixel Q(i,j), respectively.

In step S855, the weight calculator 1013 calculates the weight based onthe distance between the subject pixel and the unprocessed referencepixel, and supplies the calculated weight to the multiplier 1015. Morespecifically, if the reference pixel for which the inter-pixeldifference has been determined is the reference pixel Q(i,j), the weightcalculator 1013 calculates w_(d)=a/(i²+j²) (a is a constant) as theweight by using a coordinate parameter. It is sufficient if the weightw_(d) becomes greater as the distance is shorter and becomes smaller asthe distance is longer. Accordingly, the weight w_(d) may be calculatedby a parameter different from the parameter expressed by theabove-described equation. For example, the weight may be calculated byw_(d)=√(i²+j²) (a is a constant).

In step S856, the multiplier 1015 multiplies the inter-pixel differencesupplied from the inter-pixel difference calculator 1014 by the weightsupplied from the weight calculator 1013, and stores the multiplicationresult in the storage portion 1016. That is, in FIG. 27, the multiplier1015 multiplies the inter-pixel difference (Q(0,0)−Q(i,j)) supplied fromthe inter-pixel difference calculator 1014 by the weight w_(d) suppliedfrom the weight calculator 1013, and stores the multiplication resultw_(d)×(Q(0,0)−Q(i,j)) in the storage portion 1016.

In step S857, the inter-pixel difference calculator 1014 checks for anunprocessed reference pixel. If there is an unprocessed reference pixel,the process returns to step S854. That is, steps S854 through S857 arerepeated until all the reference pixels have been processed.

If it is determined in step S857 that there is no unprocessed referencepixel, i.e., that the differences between all the reference pixels andthe subject pixel have been determined and the weights w_(d) are setbased on the distances from the subject pixel and that themultiplication results obtained by multiplying the differences by theweights are stored in the storage portion 1016, the process proceeds tostep S858. In step S858, the inter-pixel difference calculator 1014supplies information that all the reference pixels have been processedto the maximum/minimum-value extracting portion 1017. Themaximum/minimum-value extracting portion 1017 extracts the maximum valueand the minimum value of the values stored in the storage portion 1016,and supplies the maximum and minimum values to the BEP calculator 1018.

In step S859, the BEP calculator 1018 calculates, as the BEP of thesubject pixel, the reference-pixel difference dynamic range obtained bysubtracting the minimum value from the maximum value of themultiplication results supplied from the maximum/minimum-valueextracting portion 1017, and supplies the BEP to the BFP extractingportion 932 and the feature synthesizer 933.

In step S860, the reference-pixel extracting portion 1012 determineswhether the BEPs have been calculated for all the pixels in the imagestored in the buffer 1011. If it is determined in step S860 that theBEPs have not been calculated for all the pixels, the process returns tostep S852. That is, steps S852 through S860 are repeated until the BEPshave been calculated for all the pixels of the image stored in thebuffer 1011. If it is determined in step S860 that the BEPs have beencalculated for all the pixels, the BEP extraction processing iscompleted.

According to the above-described processing, the BEP, which serves asthe feature representing an edge in a broad range of an image. That is,the BEP is indicated by the difference dynamic range of the referencepixel relative to the subject pixel. Accordingly, the BEP becomes largerif there is an edge in a broad range containing the subject pixel, andconversely, the BEP becomes smaller if there is no edge.

After the BEP extraction processing is completed, the process returns tostep S832 in FIG. 25. In step S832, the BFP extracting portion 932 ofthe broad-range feature extracting portion 911 performs BFP extractionprocessing to extract the BFP for each pixel from an HD image suppliedfrom the output phase converter 112, and supplies the extracted BFP tothe feature synthesizer 933.

The BFP extraction processing performed by the BFP extracting portion932 shown in FIG. 22 is discussed below with reference to the flowchartin FIG. 28.

In step S871, the buffer 1031 temporarily stores HD images supplied fromthe output phase converter 112.

In step S872, the reference-pixel extracting portion 1032 selects anunprocessed reference pixel as the subject pixel. In step S873, thereference-pixel extracting portion 1032 reads out the reference pixelsset for the subject pixel, and supplies the read reference pixels to theadjacent-pixel difference calculator 1033 and the weight calculator1035.

In step S874, the adjacent-pixel difference calculator 1033 calculatesthe absolute value of the difference between unprocessed referencepixels (including the subject pixel), and supplies the calculateddifference absolute value to the function transformer 1034. Morespecifically, in FIG. 29, the adjacent-pixel difference calculator 1033calculates, for example, e=|Q(i+1,j)−Q(i,j)|, as the adjacent-pixeldifference absolute value e, and supplies it to the function transformer1034. Q(i+1,j) and Q(i,j) in the adjacent-pixel difference absolutevalue represent the pixel value of the reference pixel Q(i+1,j) and thepixel value of the reference pixel Q(i,j), respectively. The circleindicated by the hatched portion in FIG. 29 represents the subject pixelQ(0,0).

The adjacent-pixel difference absolute values e of the reference pixelsare the absolute values of the differences of each pixel x indicated bya circle in FIG. 29 from the adjacent pixels in the x direction and inthe y direction. That is, as the adjacent-pixel difference absolutevalues e of the reference pixel Q(i,j), four adjacent-pixel differenceabsolute values e, such as |Q(i+1,j)−Q(i,j)|, |Q(i,j+1)−Q(i,j)|,|Q(i−1,j)−Q(i,j)|, and |Q(i,j−1)−Q(i,j)|, are set. However, sincereference pixels located at edges of an area containing the referencepixels are not adjacent to the reference pixel in one of the positiveand negative x and y directions, only three adjacent-pixel differenceabsolute values are set. Also, since reference pixels located at thefour corners in an area containing the reference pixels are not adjacentto the reference pixels in two directions of the positive and negative xand y directions, only two adjacent-pixel difference absolute values areset. As a result, if n×n reference pixels are set around the subjectpixel, 2n(n−1) adjacent-pixel difference absolute values are set.

In step S875, the function transformer 1034 transforms theadjacent-pixel difference absolute value e supplied from theadjacent-pixel difference calculator 1033 on the basis of the function fset by the BEP supplied from the BEP extracting portion 931, andsupplies the transformed adjacent-pixel difference absolute value e tothe multiplier 1036. More specifically, the function transformer 1034transforms the adjacent-pixel difference absolute value e based on thefollowing equation set from the BEP supplied from the BEP extractingportion 931, and outputs the transformed value to the multiplier 1036.

$\begin{matrix}{{f(e)} = \{ {{\begin{matrix}1 & {{{if}\mspace{14mu}{th}} \geq e} \\0 & {{{if}\mspace{14mu}{th}} < e}\end{matrix}{th}} = {{BEP}/b}} } & (18)\end{matrix}$

In equation (18), if the adjacent-pixel difference absolute value e isgreater than the threshold th=BEP/b, 0 is output to the multiplier 1036.If the adjacent-pixel difference absolute value e is smaller than orequal to the threshold th=BEP/b, 1 is output to the multiplier 1036. Inequation (18), b is a constant. The BEP used for calculating thethreshold th is the BEP of the subject pixel. It is sufficient if thethreshold th reflects the characteristic of the BEP. Accordingly, thethreshold th may be calculated by an equation different from th=BEP/b,for example, th=(√BEP)/b or th=(BEP)²/b.

In step S876, the weight calculator 1035 calculates the weight based onthe distance between the subject pixel and the center position betweenadjacent pixels for which the adjacent-pixel difference absolute valueis calculated, and supplies the resulting weight w_(e) to the multiplier1036. More specifically, if reference pixels for which theadjacent-pixel difference absolute value is calculated are referencepixels Q(i,j) and Q(i+1,j), the weight calculator 1035 calculatesw_(e)=a/((i+1/2)²+j²) (a is a constant) as the weight by using acoordinate parameter. In this case, it is sufficient if the weight w_(e)is a value determined by the distance. Accordingly, the weight w_(e) maybe calculated by an equation different from w_(e)=a/((i+1/2)²+j²), forexample, by w_(e)=a/√((i+½)²+j²) (a is a constant).

In step S877, the multiplier 1036 multiplies the adjacent-pixeldifference absolute value e supplied from the function transformer 1034by the weight w_(e) supplied from the weight calculator 1035. In stepS878, the multiplier 1036 stores the multiplication result in thestorage portion 1037. That is, in FIG. 29, the multiplier 1036multiplies the transformation result supplied from the functiontransformer 1034, i.e., 1 or 0, by the weight w_(e) supplied from theweight calculator 1035, and stores the multiplication result, i.e.,w_(e) or 0, in the storage portion 1037.

In step S879, the adjacent-pixel difference calculator 1033 checks foran unprocessed reference pixel, i.e., whether there is an unprocessedreference pixel even though an adjacent-pixel difference absolute valuehas been determined for that reference pixel. If it is determined instep S879 that there is an unprocessed reference pixel, the processreturns to step S874. Steps S874 through S879 are repeated untiladjacent-pixel difference absolute values are calculated for all thereference pixels. That is, as stated above, if the reference pixels aren×n pixels, as shown in FIG. 29, 2n(n−1) adjacent-pixel differenceabsolute values are determined. Accordingly, steps S874 through S879 arerepeated for 2n(n−1) times.

If it is determined in step S879 that there is no unprocessed referencepixel, i.e., that all the adjacent-pixel difference absolute values aredetermined and transformed by the function and weights are set based onthe distances and that the multiplication results obtained bymultiplying the absolute values by the weights are stored in the storageportion 1037, the process proceeds to step S880. In step S880, theadjacent-pixel difference calculator 1033 supplies information that allthe reference pixels have been processed to the BFP calculator 1038. TheBFP calculator 1038 calculates the sum of the multiplication resultsstored in the storage portion 1037, and outputs the sum as the BFP.

In step S881, the reference-pixel extracting portion 1032 determineswhether the BFPs have been calculated for all the pixels forming theimage stored in the buffer 1011. If it is determined in step S881 thatthe BFPs have not been calculated for all the pixels, the processreturns to step S872. That is, steps S872 through S881 are repeateduntil the BFPs have been calculated for all the pixels forming the imagestored in the buffer 1031. If it is determined in step S881 that theBFPs have been calculated for all the pixels, the BFP extractionprocessing is completed.

According to the above-described processing, the BFP is determined asthe feature that represents a flat portion in a broad area of an image.That is, if the weight w_(e) is not variable based on the distance, butis uniquely 1, the BFP becomes equal to the number of adjacent-pixeldifference absolute values smaller than or equal to the threshold th. Asa result, the BFP becomes greater if a broad range containing thesubject pixel is flat, and conversely, the BFP becomes smaller if abroad range containing the subject pixel is not flat, i.e., if itincludes many edges.

In the foregoing example, the adjacent-pixel difference absolute valuesare calculated for all the reference pixels. It is sufficient, however,if the relationship between the values of adjacent pixels can bedetermined, and thus, it is not necessary to calculate theadjacent-pixel difference absolute values for all the reference pixels.For example, the absolute values of the differences between adjacentpixels only in the x direction or in the y direction in FIG. 29 may becalculated. Alternatively, the sum of the absolute values of thedifferences between vertically adjacent pixels or between horizontallyadjacent pixels, or the sum of the absolute values of the differencesbetween vertically adjacent pixels and those between horizontallyadjacent pixels may be used as the adjacent-pixel difference absolutevalues. The same applies to processing for determining adjacent-pixeldifference absolute values in the following description.

The function transformer 1034 transforms the adjacent-pixel differenceabsolute value e by the function expressed by equation (18). However,the function is not restricted to that expressed by equation (18), andmay be a function that can transform the adjacent-pixel differenceabsolute value e into a value that can clearly show the differencebetween the adjacent-pixel difference absolute values e, as shown inFIG. 30. In this case, the effect of such a function is similar to thatwhen the function expressed by equation (18) is used.

The function f may be that shown in FIG. 30, as stated above, and mayfurther be associated with the BEP. For example, if the adjacent-pixeldifference absolute value e is greater than or equal to the thresholdth1 (th1=BEP/b1), f(e) may be A, i.e., f(e)=A, where A is a constant.Conversely, if the adjacent-pixel difference absolute value e is smallerthan or equal to the threshold th2 (th2=BEP/b2), f(e) may be B, i.e.,f(e)=B, where B is a constant. If the adjacent-pixel difference absolutevalue e is greater than the threshold th2=BEP/b2 and smaller than thethreshold th1=BEP/b1, f(e) may be (B−A)·(e−th1)/(th2−th1)+A. It issufficient if the thresholds th1 and th2 can reflect the characteristicof the BEP, the thresholds th1 and th2 may be calculated by equationsexpressed by th1=(√BEP)/b1 and th2=(√BEP)/b2, or th1=(BEP)²/b1 andth2=(BEP)²/b2.

After the BFP extraction processing is completed, the process returns tostep S833 in FIG. 25.

In step S833, the feature synthesizer 933 combines the BEP supplied fromthe BEP extracting portion 931 with the BFP supplied from the BFPextracting portion 932, and supplies the synthesized value to thebroad-range degree-of-artificiality calculator 912 as the broad-rangefeature.

In step S834, the broad-range degree-of-artificiality calculator 912performs broad-range degree-of-artificiality calculation processing onthe basis of the broad-range feature supplied from the broad-rangefeature extracting portion 911 to calculate the broad-range degree ofartificiality Art_(b), and supplies Art_(b) to thedegree-of-artificiality generator 913.

The broad-range degree-of-artificiality calculation processing performedby the broad-range degree-of-artificiality calculator 912 is describedbelow with reference to the flowchart in FIG. 31.

In step S891, the feature separator 951 obtains the broad-range featuresupplied from the broad-range feature extracting portion 911, and alsoseparates the BEP and BFP and supplies them to the broad-range boundarycomparator 952.

In step S892, the broad-range boundary comparator 952 reads outinformation concerning the broad-range boundaries from the broad-rangeboundary memory 953.

In step S893, the broad-range boundary comparator 952 selects anunprocessed pixel as the subject pixel. The broad-range boundarycomparator 952 also extracts the BEP and BFP of the subject pixel toplot the BEP and BFP on the two-dimensional BEP-BFP plane, and alsodetermines the positional relationship of the BEP and BFP to thebroad-range boundaries.

The broad-range boundaries are boundaries that can be generated from astatistical distribution obtained by plotting the BEPs and BFPsextracted from a plurality of artificial images and natural images onthe two-dimensional (BEP, BFP) space on the BEP axis and the BFP axis.The broad-range boundaries include two types of boundaries such as abroad-range artificial image boundary and a broad-range natural imageboundary. The broad-range artificial image boundary is a boundarybetween an area where only broad-range features of artificial images areplotted and an area where broad-range features of both the artificialimages and natural images are plotted. The broad-range natural imageboundary is a boundary between an area where only broad-range featuresof natural images are plotted and an area where broad-range features ofboth the artificial images and natural images are plotted. Accordingly,the two-dimensional BEP-BFP area can be divided, as shown in FIG. 32,into three types of areas, such as a broad-range artificial image area,a broad-range natural image area, and a broad-range mixture areacontaining artificial images and natural images, based on thebroad-range artificial image boundary and the broad-range natural imageboundary. In FIG. 32, the broad-range artificial image boundary and thebroad-range natural image boundary are represented by L1 and L2,respectively. The curves L1 and L2 are thus referred to as the“broad-range artificial image boundary L1” and the “broad-range naturalimage boundary L2”, respectively. In FIG. 32, the area above thebroad-range artificial image boundary L1 is the broad-range artificialimage area, the area between the broad-range artificial image boundaryL1 and the broad-range natural image boundary L2 is the broad-rangemixture area, and the area below the broad-range natural image boundaryL2 is the broad-range natural image area.

In step S894, the broad-range boundary comparator 952 determines whetherthe position at which the BEP and BFP of the subject pixel are plottedis contained in the broad-range artificial image area. For example, ifthe position at which the BEP and BFP of the subject pixel are plottedis position B1 in FIG. 32, the process proceeds to step S895. In stepS895, the broad-range boundary comparator 952 supplies information thatthe subject pixel belongs to the broad-range artificial image area tothe dividing point calculator 954.

Upon receiving this information, in step S896, the dividing pointcalculator 954 sets the broad-range degree of artificiality Art_(b) to 1and supplies Art_(b) to the degree-of-artificiality generator 913. Theprocess then proceeds to step S902.

If the position at which the BEP and BFP of the subject pixel areplotted is, for example, position B2 or B3, instead of position B1, thebroad-range boundary comparator 952 determines in step S894 that thesubject pixel does not belong to the broad-range artificial image area.The process then proceeds to step S897.

The broad-range boundary comparator 952 determines in step S897 whetherthe position at which the BEP and BFP of the subject pixel are plottedis contained in the broad-range natural image area. If the position atwhich the BEP and BFP of the subject pixel are plotted is position B3 inFIG. 32, the broad-range boundary comparator 952 determines in step S897that the subject pixel belongs to the broad-range natural image. Then,in step S898, the broad-range boundary comparator 952 suppliesinformation that the subject pixel belongs to the broad-range naturalimage area to the dividing point calculator 954.

Upon receiving this information, in step S899, the dividing pointcalculator 954 sets the broad-range degree of artificiality Art_(b) tobe 0, and supplies Art_(b) to the degree-of-artificiality generator 913.The process then proceeds to step S902.

If the position at which the BEP and BFP of the subject pixel areplotted is, for example, position B2 instead of position B3, thebroad-range boundary comparator 952 determines in step S897 that thesubject pixel belongs to neither of the broad-range artificial imagearea nor the broad-range natural image area. The process then proceedsto step S900.

In step S900, the broad-range boundary comparator 952 suppliesinformation that the subject pixel belongs to the broad-range mixturearea and information concerning the dividing point ratio of the distanceto the broad-range artificial image boundary L1 to the distance to thebroad-range natural image boundary L2 to the dividing point calculator954. That is, at position B2 in FIG. 32, the ratio of the distance tothe broad-range artificial image boundary L1 to the distance to thebroad-range natural image boundary L2 is 1−q:q. Accordingly, theinformation concerning the dividing point ratio 1−q:q is supplied to thedividing point calculator 954.

Upon receiving the information that the broad-range feature of thesubject pixel belongs to the broad-range mixture area and theinformation concerning the dividing point ratio from the broad-rangeboundary comparator 952, in step S901, the dividing point calculator 954sets the dividing point ratio q as the broad-range degree ofartificiality Art_(b), and supplies Art_(b) to thedegree-of-artificiality generator 913.

In step S902, the broad-range boundary comparator 952 checks for anunprocessed pixel, i.e., whether a determination as to whether thebroad-range feature of an unprocessed pixel belongs to the broad-rangeartificial image area, the broad-range natural image area, or thebroad-range mixture area has been made to determine the broad-rangedegree of artificiality. If it is found in step S902 that thebroad-range degrees of artificiality have not been determined for allthe pixels, the process returns to step S893.

If it is found in step S902 that the broad-range degrees ofartificiality Art_(b) have been determined for all the pixels, thebroad-range degree-of-artificiality calculation processing is completed.

According to the above-described processing, the BEPs and BFPsrepresenting the broad-range features are read out for all the pixelsand are plotted in the BEP-BFP space. Then, the position at which theBEP and BFP of a subject pixel are plotted is compared with thebroad-range artificial image boundary and the broad-range natural imageboundary that are statistically determined from the distribution of aplurality of artificial images and natural images. Based on thepositional relationship of the plotted position to the broad-rangeartificial image boundary and the broad-range natural image boundary,the broad-range degree of artificiality Art_(b) is calculated. In theabove-described example, when the subject pixel is contained in thebroad-range mixture area, the dividing point ratio q is set as thedegree of artificiality Art_(b). However, since it is sufficient if thedividing point ratio reflects the characteristic of the broad-rangedegree of artificiality Art_(b), q² or √q may be used as the degree ofartificiality Art_(b).

After the broad-range degree-of-artificiality processing is completed,the process returns to step S835 in FIG. 25.

In step S835, the PNDP extracting portion 971 of the narrow-rangefeature extracting portion 912 performs PNDP processing to extract thePNDP for each pixel from an HD image supplied from the output phaseconverter 112, and supplies the extracted PNDP to the featuresynthesizer 933.

The PNDP extraction processing performed by the PDNP extracting portion971 shown in FIG. 23 is described below with reference to the flowchartin FIG. 33.

In step S911, the buffer 1051 temporarily stores HD images supplied fromthe output phase converter 112.

In step S912, the reference-pixel extracting portion 1052 selects anunprocessed pixel as the subject pixel. In step S913, thereference-pixel extracting portion 1052 reads out the reference pixelsset for the subject pixel and supplies the reference pixels to thelong-tap extracting portion 1053.

In step S914, the long-tap extracting portion 1053 extracts a long tapfrom the reference pixels. In step S915, the long-tap extracting portion1053 stores a pixel value forming the long tap in the pixel-valuestorage portion 1054, and also supplies the pixel value forming the longtap to the SNDP extracting portion 972. The long tap includes pixelscontained in a predetermined area set for the subject pixel andcontained in the reference pixels, e.g., m×m pixels around the subjectpixel, as shown in FIG. 34. In FIG. 34, an n-pixel×n-pixel area (n>m)indicates a reference pixel range. In FIG. 34, the pixels P(i,j) formingthe long tap are indicated by the white circles on the two-dimensionalcoordinates in the x direction and in the y direction around the subjectpixel P(0,0) indicated by the hatched portion in the image. The pixelsP(i,j) contained in the area surrounded by the broken lines (m×m) aroundthe subject pixel Q(0,0) forming the long tap can be represented by−(m−1)/2≦i≦(m−1)/2, −(m−1)/2≦j≦(m−1)/2, where i and j are integers.

In step S916, the pixel value storage portion 1054 checks for anunprocessed pixel forming the long tap. If an unprocessed pixel formingthe long tap is found in step S916, the process returns to step S915.That is, steps S915 and S916 are repeated until all the pixel valuesforming the long tap are processed.

If it is found in step S916 that there is no unprocessed pixel formingthe long tap, i.e., that all the pixel values of the long tap have beenextracted and stored in the pixel-value storage portion 1054, theprocess proceeds to step S917. In step S917, the pixel-value extractingportion 1054 supplies information that all the pixels forming the longtap have been processed to the maximum/minimum-value extracting portion1055. Upon receiving this information, the maximum/minimum-valueextracting portion 1055 extracts the maximum value and the minimum valuefrom the pixel values stored in the pixel-value storage portion 1054,and supplies the maximum and minimum values to the PNDP calculator 1056.

In step S918, the PNDP calculator 1056 calculates, as the PNDP for thesubject pixel, the pixel-value dynamic range of the long tap obtained bysubtracting the minimum value from the maximum value supplied from themaximum/minimum-value extracting portion 1055, and supplies the PNDP tothe feature synthesizer 933.

In step S919, the reference-pixel extracting portion 1052 determineswhether the PNDPs have been calculated for all the pixels in the imagestored in the buffer 1051. If it is found in step S919 that the PNDPshave not been calculated for all the pixels, the process returns to stepS912. That is, steps S912 through S919 are repeated until the PNDPs havebeen determined for all the pixels. If it is found in step S919 that thePNDPs have been determined for all the pixels, the PNDP extractionprocessing is completed.

According to the above-described processing, the PNDP representing thefeature of an edge in a narrow area of an image is determined. That is,the PNDP represents the dynamic range of the pixel values forming a longtap set for the subject pixel. Accordingly, the PNDP becomes greater ifthere is an edge in a narrow area containing the subject pixel, andconversely, the PNDP becomes smaller if there is no edge.

After the PNDP extraction processing, the process returns to step S836in FIG. 25.

In step S836, the SNDP extracting portion 972 of the narrow-rangefeature extracting portion 914 performs SNDP extraction processing toextract an SNDP for each pixel from an HD image supplied from the outputphase converter 112, and supplies the extracted SNDP to the featuresynthesizer 933.

The SNDP extraction processing performed by the SNDP extracting portion972 shown in FIG. 24 is discussed below with reference to the flowchartin FIG. 35.

In step S941, the buffer 1071 temporarily stores HD images supplied fromthe output phase converter 112.

In step S942, the reference-pixel extracting portion 1072 selects anunprocessed pixel from the buffer 1051 as the subject pixel. In stepS943, the reference-pixel extracting portion 1072 reads out referencepixels set for the subject pixel and supplies the reference pixels tothe short-tap extracting portion 1073.

In step S944, the short-tap extracting portion 1073 extracts a short tapfrom the reference pixels on the basis of the reference pixels and theinformation concerning the long tap supplied from the PNDP extractingportion 971. The short tap includes a plurality of pixels contained inthe long tap and including the subject pixel, for example, pixels withina predetermined area which is set for the subject pixel and which iscontained in the long tap of the reference pixels. For example, theshort tap may be a short tap ST1, as indicated by pattern A in FIG. 36,including a total of five pixels composed of a subject pixel and fourpixels adjacent to the subject pixel in the horizontal and verticaldirection. The short tap may be a short tap ST2, as indicated by patternB in FIG. 36, including a total of nine (3×3) pixels composed of thesubject pixel at the bottom right and eight pixels, or may be a shorttap ST3, as indicated by pattern C in FIG. 36, including nine (3×3)pixels composed of the subject pixel at the top left and eight pixels.Alternatively, the short tap may be a short tap ST3, as indicated bypattern D in FIG. 36, including nine (3×3) pixels composed of thesubject pixel at the bottom left and eight pixels, or may be a short tapST5, as indicated by pattern E in FIG. 36, including nine (3×3) pixelscomposed of the subject pixel at the top right and eight pixels. Aplurality of patterns of short taps are referred to as a “short tapset”. Accordingly, if short taps ST1 through ST5 shown in FIG. 36 areused, it means that a short tap set including five patterns of shorttaps is formed. In FIG. 36, the circles indicate pixels among which thecircles indicated by the hatched portions represent the subject pixels.The pixels surrounded by the solid lines designate the pixels formingthe short taps.

In step S945, the short-tap extracting portion 1073 extracts the pixelvalue of an unprocessed pixel forming a short tap and stores theextracted pixel value in the pixel value storage portion 1074.

In step S946, the short-tap extracting portion 1073 checks for anunprocessed pixel forming the short tap. If there is an unprocessedpixel forming the short tap, the process returns to step S945. That is,steps S945 and S946 are repeated for all the pixels contained in theshort tap.

If it is found in step S946 that there is no unprocessed pixel formingthe short tap, i.e., that the pixel values of all the pixels forming oneshort tap are stored in the pixel value storage portion 1074, theprocess proceeds to step S947. In step S947, the pixel value extractingportion 1072 supplies information that the pixel values of all thepixels forming the short tap have been stored to themaximum/minimum-value extracting portion 1075. The maximum/minimum-valueextracting portion 1075 extracts the maximum value and the minimum valueof the pixel values of the short tap stored in the pixel value storageportion 1074, and supplies the maximum and minimum values to the DRcalculator 1076.

In step S948, the DR calculator 1076 subtracts the minimum value fromthe maximum value supplied from the maximum/minimum-value extractingportion 1075 to calculate the dynamic range of the pixel values of theshort tap as the pixel value DR for the subject pixel. In step S949, theDR calculator 1076 stores the calculated pixel value DR in the DRstorage portion 1077.

In step S950, the short-tap extracting portion 1073 checks for anunprocessed short tap. If there is an unprocessed short tap, the processreturns to step S944. That is, steps S944 through S950 are repeateduntil DRs have been calculated for all the short taps forming the shorttap set. Accordingly, for example, if the short tap set includes fivepatterns, as shown in FIG. 36, steps S944 through S950 are repeated forfive times.

If it is determined in step S950 that all the short taps have beenextracted, i.e., that DRs have been calculated for all the short taps,the process proceeds to step S951. In step S951, the short-tapextracting portion 1073 supplies information that DRs have beencalculated for all the short taps to the SNDP selector 1078. The SNDPselector 1078 compares the DRs of the short taps stored in the DRstorage portion 1077 with each other to select the minimum DR as theSNDP and supplies the SNDP to the feature synthesizer 973.

In step S952, the reference-pixel extracting portion 1072 determineswhether SNDPs have been calculated for all the pixels in the imagestored in the buffer 1071. If it is found in step S952 that SNDPs havenot been calculated for all the pixels, the process returns to stepS942. That is, steps S942 through S952 are repeated until SNDPs havebeen determined for all the pixels of the image stored in the buffer1071. If it is determined in step S952 that SNDPs have been calculatedfor all the pixels, the SNDP extraction processing is completed.

According to the above-described processing, the SNDP, which is thefeature representing the flatness of a flat portion in a narrow range ofan image, is determined for each pixel forming the image. That is, theSNDP, which is the minimum DR of the pixel value DRs of a plurality ofshort taps contained in a long tap for a subject pixel, represents theflatness of the most flat portion of the short taps contained in thelong tap. Accordingly, the SNDP becomes smaller if the flat portioncontained in a narrow area including the subject pixel is more flat, andconversely, the SNDP becomes greater if the flat portion is less flat.

After the SNDP extraction processing, the process returns to step S837in FIG. 25.

In step S837, the feature synthesizer 973 combines the PNDP suppliedfrom the PNDP extracting portion 971 with the SNDP supplied from theSNDP extracting portion 972 and supplies the resulting synthesized valueto the narrow-range degree-of-artificiality calculator 915 as thenarrow-range feature.

In step S838, the narrow-range degree-of-artificiality calculator 915performs narrow-range degree-of-artificiality calculation processing onthe basis of the narrow-range feature supplied from the narrow-rangefeature extracting portion 914 to calculate the narrow-range degree ofartificiality Art_(n), and supplies Art_(n) to thedegree-of-artificiality generator 913.

The narrow-range degree-of-artificiality calculation processingperformed by the narrow-range degree-of-artificiality calculator 915 isdiscussed below with reference to the flowchart in FIG. 37.

In step S971, the feature separator 991 obtains a narrow-range featuresupplied from the narrow-range feature extracting portion 914 toseparate the PNDP and SNDP and supplies the separated PDNP and SNDP tothe narrow-range boundary comparator 992.

In step S972, the narrow-range boundary comparator 992 reads outinformation concerning the narrow-range boundaries from the narrow-rangeboundary memory 993.

In step S973, the narrow-range boundary comparator 992 selects anunprocessed pixel as the subject pixel and extracts the PNDP and SNDP ofthe subject pixel to plot the extracted PNDP and SNDP on thetwo-dimensional PNDP-SNDP plane. The narrow-range boundary comparator992 also compares the position at which the PNDP and SNDP are plottedwith the read narrow-range boundaries.

The narrow-range boundaries are boundaries that can be generated from astatistical distribution obtained by plotting the PNDPs and SNDPsextracted from a plurality of artificial images and natural images onthe two-dimensional (PNDP, SNDP) space on the PDNP axis and the SNDPaxis. The narrow-range boundaries include two types of boundaries suchas a narrow-range artificial image boundary and a narrow-range naturalimage boundary. The narrow-range artificial image boundary is a boundarybetween an area where only narrow-range features of artificial imagesare plotted and an area where narrow-range features of both theartificial images and natural images are plotted. The narrow-rangenatural image boundary is a boundary between an area where onlynarrow-range features of natural images are plotted and an area wherenarrow-range features of both the artificial images and natural imagesare plotted. Accordingly, the two-dimensional PNDP-SNDP area can bedivided, as shown in FIG. 38, into three types of areas, such as anarrow-range artificial image area, a narrow-range natural image area,and a narrow-range mixture area containing artificial images and naturalimages, based on the narrow-range artificial image boundary and thenarrow-range natural image boundary. In FIG. 38, the narrow-rangeartificial image boundary and the narrow-range natural image boundaryare represented by L11 and L12, respectively. The curves L11 and L12 arethus referred to as the “narrow-range natural image boundary L11” andthe “narrow-range artificial image boundary L12”, respectively. In FIG.38, the area above the narrow-range natural image boundary L11 is thenarrow-range natural image area, the area between the narrow-rangenatural image boundary L11 and the narrow-range artificial imageboundary L12 is the narrow-range mixture area, and the area below thenarrow-range artificial image boundary L12 is the narrow-rangeartificial image area.

In step S974, the narrow-range boundary comparator 992 determineswhether the position at which the PNDP and SNDP of the subject pixel areplotted is contained in the narrow-range artificial image area. Forexample, if the position at which the PNDP and SNDP of the subject pixelare plotted is position N3 in FIG. 38, the process proceeds to stepS975. In step S975, the narrow-range boundary comparator 992 suppliesinformation that the subject pixel belongs to the narrow-rangeartificial image area to the dividing point calculator 994.

Upon receiving this information, in step S976, the dividing pointcalculator 994 sets the narrow-range degree of artificiality Art_(n) to1 and supplies Art_(n) to the degree-of-artificiality generator 913. Theprocess then proceeds to step S982.

If the position at which the PNDP and SNDP of the subject pixel areplotted is, for example, position N2 or N1, instead of position N3, thenarrow-range boundary comparator 992 determines in step S974 that thesubject pixel does not belong to the narrow-range artificial image area.The process then proceeds to step S977.

The narrow-range boundary comparator 992 determines in step S977 whetherthe position at which the PNDP and SNDP of the subject pixel are plottedis contained in the narrow-range natural image area. If the position atwhich the PNDP and SNDP of the subject pixel are plotted is position N1in FIG. 38, the narrow-range boundary comparator 992 determines in stepS977 that the subject pixel belongs to the narrow-range natural image.Then, in step S978, the narrow-range boundary comparator 992 suppliesinformation that the subject pixel belongs to the narrow-range naturalimage area to the dividing point calculator 994.

Upon receiving this information, in step S979, the dividing pointcalculator 994 sets the narrow-range degree of artificiality Art_(n) tobe 0, and supplies Art_(n) to the degree-of-artificiality generator 913.The process then proceeds to step S982.

If the position at which the PNDP and SNDP of the subject pixel areplotted is, for example, position N2 instead of position N1, thenarrow-range boundary comparator 992 determines in step S977 that thesubject pixel belongs to neither of the narrow-range artificial imagearea nor the narrow-range natural image area. The process then proceedsto step S980.

In step S980, the narrow-range boundary comparator 992 suppliesinformation that the subject pixel belongs to the narrow-range mixturearea and information concerning the dividing point ratio of the distanceto the narrow-range artificial image boundary L11 to the distance to thenarrow-range natural image boundary L12 to the dividing point calculator994. That is, at position N2 in FIG. 38, the ratio of the distance tothe narrow-range natural image boundary L11 to the distance to thenarrow-range artificial image boundary L12 is p:1−p. Accordingly, theinformation concerning the dividing point ratio p:1−p is supplied to thedividing point calculator 994.

Upon receiving the information that the subject pixel belongs to thenarrow-range mixture area and the information concerning the dividingpoint ratio from the narrow-range boundary comparator 992, in step S981,the dividing point calculator 994 sets the dividing point ratio p as thenarrow-range degree of artificiality Art_(n), and supplies Art_(n) tothe degree-of-artificiality generator 913.

In step S982, the narrow-range boundary comparator 992 checks for anunprocessed pixel, i.e., whether a determination as to whether thenarrow-range feature of an unprocessed pixel belongs to the narrow-rangeartificial image area, the narrow-range natural image area, or thenarrow-range mixture area has been made to determine the narrow-rangedegree of artificiality. If it is found in step S982 that thenarrow-range degrees of artificiality have not been determined for allthe pixels, the process returns to step S973.

If it is found in step S982 that the narrow-range degrees ofartificiality Art_(n) have been determined for all the pixels, thenarrow-range degree-of-artificiality calculation processing iscompleted.

According to the above-described processing, the PNDPs and SNDPsrepresenting the narrow-range features are read out for all the pixelsand are plotted in the PNDP-SNDP space. Then, the position at which thePNDP and SNDP of a subject pixel are plotted is compared with thenarrow-range artificial image boundary and the narrow-range naturalimage boundary that are statistically determined from the distributionof a plurality of artificial images and natural images. Based on thepositional relationship of the plotted position to the narrow-rangeartificial image boundary and the narrow-range natural image boundary,the narrow-range degree of artificiality Art_(n) is calculated. In theabove-described example, when the subject pixel is contained in thenarrow-range mixture area, the dividing point ratio p is set as thedegree of artificiality Art_(n). However, since it is sufficient if thedividing point ratio reflects the characteristic of the narrow-rangedegree of artificiality Art_(n), p² or √p may be used as the degree ofartificiality Art_(n).

After the narrow-range degree-of-artificiality calculation processing,the process returns to step S839 in FIG. 25.

In step S839, the degree-of-artificiality generator 913 calculates thefollowing equation based on the broad-range degree of artificialityArt_(b) supplied from the broad-range degree-of-artificiality calculator912 and the narrow-range degree of artificiality Art_(n) supplied fromthe narrow-range degree-of-artificiality calculator 915 to generate thedegree of artificiality Art, and supplies Art to the image processor113:Art=α·Art_(n)+β·Art_(b)+γ·Art_(n)Art_(b)  (19)where α, β, and γ are constants. The degree of artificiality is set,assuming that 0≦Art≦1. Accordingly, if the degree of artificiality Artis found to be greater than 1, the degree-of-artificiality generator 913clips the degree of artificiality Art to be 1 and supplies Art to theimage processor 113.

According to the above-described processing, in step S7 in FIG. 2, thesynthesizer 133 calculates the following equation to combine the pixelvalues of the pixels forming the high-quality natural image suppliedfrom the natural-image prediction unit 131 with those forming thehigh-quality artificial image supplied from the artificial-imageprediction unit 132 in accordance with the degree of artificiality:Pix=Pix_(art)·Art+Pix_(nat)·(1−Art)  (20)where Pix indicates the pixel value of each pixel forming the finalhigh-quality image; Pix_(nat) designates the natural image supplied fromthe natural-image prediction unit 131; Pix_(art) represents theartificial image supplied from the artificial-image prediction unit 132;and Art represents the coefficient of the degree of artificiality.

According to the foregoing description, pixels subjected to imageprocessing as an artificial image and pixels subjected to imageprocessing as a natural image are combined in accordance with thedegrees of artificiality. Thus, it is possible to individually performoptimal processing on an area of natural image components and an area ofartificial image components by distinguishing the two areas from eachother.

As in the BEP extraction processing discussed with reference to theflowchart in FIG. 26, the difference between the maximum value and theminimum value of the differences between a subject pixel and referencepixels is determined as the difference dynamic range of the subjectpixel. It is sufficient, however, if a portion where the pixel value ofreference pixels changes sharply is determined. Accordingly, instead ofthe difference dynamic range, the dynamic range of the pixel values ofpixels contained in a reference pixel area may be determined. If thepixel-value dynamic range is used as the BEP, the BEP value may changegreatly depending on whether there is a reference pixel contained in anedge at or near a boundary of a reference pixel area. As stated above,therefore, the difference dynamic range obtained by applying a weight tothe difference between a subject pixel and a reference pixel accordingto the distance therebetween may preferably be used.

To determine the BEPs, the difference absolute values between the pixelvalues of adjacent pixels for all the reference pixels may be rearrangedin ascending order, and then, higher difference absolute values may bedetermined as the BEPs.

FIG. 39 illustrates the configuration of a BEP extracting portion thatdetermines BEPs in this manner.

The BEP extracting portion 931 shown in FIG. 39 includes a buffer 1101,a reference-pixel extracting portion 1102, an adjacent-pixel differenceabsolute value calculator 1103, a rearranging portion 1104, ahigher-level extracting portion 1105, an a BEP calculator 1106.

The buffer 1101, which is basically similar to the buffer 1011 shown inFIG. 21, temporarily stores HD images supplied from the output phaseconverter 112 and supplies an HD image to the reference-pixel extractingportion 1102 if necessary. The reference-pixel extracting portion 1102,which is basically similar to the reference-pixel extracting portion1021 shown in FIG. 21, sequentially reads out reference pixels for eachsubject pixel and supplies the reference pixels to the adjacent-pixeldifference absolute value calculator 1103.

The adjacent-pixel difference absolute value calculator 1103 determinesthe difference absolute values of the pixel values between verticallyand horizontally adjacent reference pixels, and supplies the calculateddifference absolute values to the rearranging portion 1104. Therearranging portion 1104 rearranges the difference absolute valuessupplied from the adjacent-pixel difference absolute value calculator1103 in ascending order, and supplies the rearranged difference absolutevalues to the higher-level extracting portion 1105.

The higher-level extracting portion 1105 extracts the higher N1-ththrough N2-th values based on information concerning the adjacent-pixeldifference absolute values rearranged in ascending order supplied fromthe rearranging portion 1104, and supplies the extracted higher N1-ththrough N2-th values to the BEP calculator 1106. The BEP calculator 1106determines the average of the higher N1-th through N2-th values as theBEP based on the information concerning the rearranged adjacent-pixeldifference absolute values supplied from the higher-level extractingportion 1105, and supplies the BEP to the feature synthesizer 933.

The BEP extraction processing performed by the BEP extracting portion931 shown in FIG. 39 is described below with reference to the flowchartin FIG. 40.

In step S1001, the buffer 1101 temporarily stores HD images suppliedfrom the output phase converter 112.

In step S1002, the reference-pixel extracting portion 1102 selects anunprocessed pixel from the buffer 1101 as the reference pixel. In stepS1003, the reference-pixel extracting portion 1102 reads out referencepixels set for the subject pixel and supplies the reference pixels tothe adjacent-pixel difference absolute value calculator 1103.

In step S1004, the adjacent-pixel difference absolute value calculator1103 calculates the difference absolute value between unprocessedadjacent reference pixels (including the subject pixel), and suppliesthe calculated difference absolute value to the rearranging portion1104.

In step S1005, the rearranging portion 1104 stores the adjacent-pixeldifference pixel value supplied from the adjacent-pixel differenceabsolute value calculator 1103.

In step S1006, the adjacent-pixel difference absolute value calculator1103 determines whether the difference absolute values between adjacentpixel values for all the reference pixels have been calculated. If it isdetermined in step S1006 that the difference absolute values have notbeen calculated for all the adjacent reference pixels, the processreturns to step S1004.

That is, steps S1004 through S1006 are repeated until the differenceabsolute values for all the adjacent reference pixels have beendetermined. If it is determined in step S1006 that the differenceabsolute values have been calculated for all the adjacent referencepixels, the process proceeds to step S1007. In step S1007, therearranging portion 1104 rearranges the calculated adjacent-pixeldifference absolute values in ascending order, and supplies therearranged absolute values to the higher-level extracting portion 1105.For example, the rearranging portion 1104 rearranges the adjacent-pixeldifference absolute values in ascending order, as shown in FIG. 41, andsupplies the rearranged absolute values to the higher-level extractingportion 1105. In FIG. 41, the horizontal axis represents the order andthe vertical axis indicates the adjacent-pixel difference absolutevalue.

In step S1008, the higher-level extracting portion 1105 extracts thehigher N1-th through N2-th absolute values from the absolute valuesrearranged in ascending order, and supplies the higher N1-th throughN2-th absolute values to the BEP calculator 1106. That is, in the caseof FIG. 41, the higher N1-th through N2-th adjacent-pixel differenceabsolute values are extracted.

In step S1009, the BEP calculator 1106 calculates the average of thehigher N1-th through N2-th adjacent-pixel difference absolute values asthe BEP, and outputs the BEP to the feature synthesizer 933.

In step S1010, the reference-pixel extracting portion 1102 determineswhether BEPs have been calculated for all the pixels of the image storedin the buffer 1101. If it is found in step S1010 that BEPs have not beencalculated for all the pixels, the process returns to step S1002. Thatis, steps S1002 through S1010 are repeated until BEPs have beencalculated for all the pixels. If it is found in step S1010 that BEPshave been determined for all the pixels, the BEP extraction processingis completed.

According to the above-described processing, the BEP representing thefeature of an edge in a broad area of an image is determined. That is,the BEP is the average of the higher N1-th and N2-th adjacent-pixeldifference absolute values, and a relatively large value of theadjacent-pixel difference absolute values of the reference pixels isextracted as the feature representing an edge in a broad range of theimage.

In the above-described example, the average of the higher N1-th andN2-th absolute values is used as the BEP. It is sufficient, however, ifthe feature of a broad-range edge using adjacent-pixel differenceabsolute values of reference pixels is determined. Accordingly, insteadof the average of higher absolute values, the product sum of weightedadjacent-pixel difference absolute values may be used. Alternatively,the N-th absolute value between the higher N1-th and N2-th absolutevalues may be directly used as the BEP, as shown in FIG. 41.

In the above-described example, as the BFP extraction processingdiscussed with reference to the flowchart in FIG. 28, the adjacent-pixeldifference absolute value of each reference pixel is compared with thepredetermined threshold th, and values larger than the threshold th aretransformed into a predetermined value by a function, and the sum of thevalues of all the reference pixels is used as the BFP. It is sufficient,however, if a portion where the pixel value of the reference pixelschanges very little is determined. Accordingly, the difference absolutevalues of adjacent reference pixels may be rearranged in ascendingorder, and the lower difference absolute values may be used as the BFP.

FIG. 42 illustrates the configuration of a BFP extraction portion thatdetermines BFPs in this manner.

The BFP extracting portion 932 shown in FIG. 42 includes a buffer 1121,a reference-pixel extracting portion 1122, an adjacent-pixel differenceabsolute value calculator 1123, a rearranging portion 1124, alower-level extracting portion 1125, and a BFP calculator 1126.

The buffer 1121, which is basically similar to the buffer 1031 shown inFIG. 22, temporarily stores HD images supplied from the output phaseconverter 112 and sequentially supplies the HD images to thereference-pixel extracting portion 1122 if necessary. Thereference-pixel extracting portion 1122, which is basically similar tothe reference-pixel extracting portion 1031 shown in FIG. 22,sequentially reads out reference pixels for each subject pixel andsupplies the reference pixels to the adjacent-pixel difference absolutevalue calculator 1123.

The adjacent-pixel difference absolute value calculator 1123 determinesthe difference absolute values between vertically and horizontallyadjacent reference pixels, and supplies the calculated differenceabsolute values to the rearranging portion 1124. The rearranging portion1124 rearranges the difference absolute values supplied from theadjacent-pixel difference absolute value calculator 1123 in ascendingorder, and supplies the rearranged difference absolute values to thelower-level extracting portion 1125.

The lower-level extracting portion 1125 extracts the lower n1-th throughn2-th values based on information concerning the adjacent-pixeldifference absolute values rearranged in ascending order supplied fromthe rearranging portion 1104, and supplies the extracted lower n1-ththrough n2-th values to the BFP calculator 1126. The BFP calculator 1126determines the average of the lower n1-th through n2-th values as theBFP based on the information concerning the rearranged adjacent-pixeldifference absolute values supplied from the lower-level extractingportion 1125, and supplies the BFP to the feature synthesizer 933.

The BFP extraction processing performed by the BFP extracting portion932 shown in FIG. 42 is described below with reference to the flowchartin FIG. 43.

In step S1031, the buffer 1121 temporarily stores HD images suppliedfrom the output phase converter 112.

In step S1032, the reference-pixel extracting portion 1122 selects anunprocessed pixel from the buffer 1121 as the reference pixel. In stepS1033, the reference-pixel extracting portion 1122 reads out referencepixels set for the subject pixel and supplies the reference pixels tothe adjacent-pixel difference absolute value calculator 1123.

In step S1034, the adjacent-pixel difference absolute value calculator1123 calculates the difference absolute value between unprocessedadjacent reference pixels (including the subject pixel), and suppliesthe calculated difference absolute value to the rearranging portion1124.

In step S1035, the rearranging portion 1124 stores the adjacent-pixeldifference pixel value supplied from the adjacent-pixel differenceabsolute value calculator 1123.

In step S1036, the adjacent-pixel difference absolute value calculator1123 determines whether the difference absolute values between adjacentpixel values for all the reference pixels have been calculated. If it isdetermined in step S1036 that the difference absolute values have notbeen calculated for all the adjacent reference pixels, the processreturns to step S1034.

That is, steps S1034 through S1036 are repeated until the differenceabsolute values for all the adjacent reference pixels have beendetermined. If it is determined in step S1036 that the differenceabsolute values have been calculated for all the adjacent referencepixels, the process proceeds to step S1037. In step S1037, therearranging portion 1124 rearranges the calculated adjacent-pixeldifference absolute values in ascending order, and supplies therearranged absolute values to the lower-level extracting portion 1125.For example, the rearranging portion 1124 rearranges the adjacent-pixeldifference absolute values in ascending order, as shown in FIG. 44, andsupplies the rearranged absolute values to the lower-level extractingportion 1125. In FIG. 44, the horizontal axis represents the order andthe vertical axis indicates the adjacent-pixel difference absolutevalue.

In step S1038, the lower-level extracting portion 1125 extracts thelower n1-th through n2-th absolute values from the absolute valuesrearranged in ascending order, and supplies the lower n1-th throughn2-th absolute values to the BFP calculator 1126. That is, in the caseof FIG. 44, the lower n1-th through n2-th adjacent-pixel differenceabsolute values are extracted.

In step S1039, the BFP calculator 1126 calculates the average of thelower n1-th through n2-th adjacent-pixel difference absolute values asthe BFP, and outputs the BFP to the feature synthesizer 933.

In step S1040, the reference-pixel extracting portion 1122 determineswhether BFPs have been calculated for all the pixels of the image storedin the buffer 1121. If it is found in step S1040 that BFPs have not beencalculated for all the pixels, the process returns to step S1032. Thatis, steps S1032 through S1040 are repeated until BFPs have beencalculated for all the pixels. If it is found in step S1040 that BFPshave been determined for all the pixels, the BFP extraction processingis completed.

According to the above-described processing, the BFP representing theflatness of a flat portion in a broad area of an image is determined.That is, the BFP is the average of the lower n1-th and n2-thadjacent-pixel difference absolute values, and a relatively small valueof the adjacent-pixel difference absolute values of the reference pixelsis extracted as the feature representing a flat portion in a broad rangeof the image.

In the above-described example, the average of the lower n1-th and n2-thabsolute values is used as the BFP. It is sufficient, however, if thefeature in a broad range using adjacent-pixel difference absolute valuesof reference pixels is determined. Accordingly, instead of the averageof lower absolute values, the product sum of weighted adjacent-pixeldifference absolute values may be used. Alternatively, the n-th absolutevalue between the lower n1-th and n2-th absolute values may be directlyused as the BFP, as shown in FIG. 44.

As described above, as in the PNDP extraction processing discussed withreference to the flowchart in FIG. 33, the PNDP is determined from thepixel-value dynamic range of a long tap contained in reference pixels.Alternatively, the pixel-value dynamic range of reference pixels may beused as the PNDP.

FIG. 45 illustrates the configuration of a PDNP extracting portion thatdetermines PNDPs in this manner.

The PNDP extracting portion 971 shown in FIG. 45 includes a buffer 1141,a reference-pixel extracting portion 1142, a pixel value storage portion1143, a maximum/minimum-value extracting portion 1144, and a PNDPcalculator 1145.

The buffer 1141 temporarily stores HD images supplied from the outputphase converter 112, and sequentially supplies the HD images to thereference-pixel extracting portion 1142 if necessary. Thereference-pixel extracting portion 1142 sequentially reads out referencepixels for each subject pixel, and stores them in the pixel valuestorage portion 1143.

The maximum/minimum-value extracting portion 1144 extracts the maximumvalue and the minimum value of the pixel values of all the referencepixels stored in the pixel value storage portion 1143 and supplies themaximum and minimum values to the PNDP calculator 1145.

The PNDP calculator 1145 determines the pixel-value dynamic range of thereference pixels by subtracting the minimum value from the maximum valueof the pixel values of all the reference pixels supplied from themaximum/minimum-value extracting portion 1144, and outputs the dynamicrange as the PNDP of the subject pixel.

The PNDP extraction processing performed by the PNDP extracting portion971 shown in FIG. 45 is described below with reference to the flowchartin FIG. 46.

In step S1051, the buffer 1141 temporarily stores HD images suppliedfrom the output phase converter 112.

In step S1052, the reference-pixel extracting portion 1142 selects anunprocessed pixel from the buffer 1141 as the subject pixel. In stepS1053, the reference-pixel extracting portion 1142 reads out referencepixels set for the subject pixel. In step S1054, the reference-pixelextracting portion 1142 stores the pixel value of the reference pixel inthe pixel value storage portion 1143.

In step S1055, the reference-pixel extracting portion 1142 checks for anunprocessed pixel, i.e., whether the pixel values of all the referencepixels are stored in the pixel value storage portion 1143. If there isan unprocessed reference pixel, the process returns to step S1054. Thatis, steps S1054 and S1055 are repeated until all the reference pixelshave been processed.

If it is determined in step S1055 that there is no unprocessed pixel,i.e., the pixel values of all the reference pixels are extracted andstored in the pixel value storage portion 1143, the process proceeds tostep S1056. In step S1056, the reference-pixel extracting portion 1142supplies information that all the reference pixels have been processedto the maximum/minimum-value extracting portion 1144. Upon receivingthis information, the maximum/minimum-value extracting portion 1144extracts the maximum value and the minimum value of the pixel valuesstored in the pixel value storage portion 1054, and supplies theextracted maximum and minimum values to the PNDP calculator 1145.

In step S1057, the PNDP calculator 1145 calculates the pixel-valuedynamic range as the PDNP for the subject pixel by subtracting theminimum value from the maximum value of the pixel value supplied fromthe maximum/minimum-value extracting portion 1144, and supplies the PNDPto the feature synthesizer 933.

In step S1058, the reference-pixel extracting portion 1142 determineswhether PNDPs have been calculated for all the pixels of the imagestored in the buffer 1141. If it is found in step S1058 that PNDPs havenot been calculated for all the pixels, the process returns to stepS1052. That is, steps S1052 through S1058 are repeated until PNDPs havebeen determined for all the pixels of the image stored in the buffer1141. If it is determined in step S1058 that PNDPs have been calculatedfor all the pixels, the PNDP extraction processing is completed.

According to the above-described processing, the PNDP representing thefeature of an edge in a narrow area of an image is determined. That is,the PNDP represents the pixel-value dynamic range of the referencepixels for a subject pixel. Accordingly, the PNDP becomes greater ifthere is an edge in a narrow area containing the subject pixel, andconversely, the PNDP becomes smaller if there is no edge.

As in the SNDP extraction processing discussed with reference to theflowchart in FIG. 35, the minimum dynamic range of the pixel-valuedynamic ranges of short taps is extracted as the SNDP. It is sufficient,however, if the feature reflects the characteristic of a flat portion.Accordingly, as in the BFP extracting portion 932 shown in FIG. 22, thedifference absolute value between a reference pixel and a subject pixelmay be transformed by a function, and the transformed value may becompared with a threshold in accordance with a weight determined fromthe path from the reference pixel to the subject pixel. The comparisonresults may then be added and the added value may be extracted as theSNDP.

FIG. 47 illustrates the configuration of an SNDP extracting portion thatdetermines SNDPs in this manner.

The SNDP extracting portion 972 shown in FIG. 47 includes a buffer 1161,a reference-pixel extracting portion 1162, an inter-pixel differencecalculator 1163, a function transformer 1164, a weight calculator 1165,a multiplier 1166, a storage portion 1167, and an SNDP calculator 1168.

The buffer 1161 temporarily stores HD images supplied from the outputphase converter 112, and sequentially supplies the HD images to thereference-pixel extracting portion 1162 if necessary. Thereference-pixel extracting portion 1162 sequentially reads out referencepixels for each subject pixel, and supplies the reference pixels to theinter-pixel difference calculator 1163 and the weight calculator 1165.

The inter-pixel difference calculator 1163 supplies the differenceabsolute value between the subject pixel and each of the referencepixels supplied from the reference-pixel extracting portion 1162 to thefunction transformer 1164. The function transformer 1164 transforms theinter-pixel difference absolute value supplied from the inter-pixeldifference calculator 1163 by a predetermined function, and supplies thetransformed value to the multiplier 1166.

The weight calculator 1165 includes an interpolated pixel generator 1165a and a difference calculator 1165 b, and calculates, for each referencepixel supplied from the reference-pixel extracting portion 1162, aweight in accordance with the distance from the subject pixel, andsupplies the calculated weight to the multiplier 1166. Morespecifically, the weight calculator 1165 controls the differencecalculator 1165 b to accumulate the difference absolute values betweenpixels located on the path from the reference pixel to the subject pixelto determine the weight w_(s) based on the accumulation result. In thiscase, the weight calculator 1165 controls the interpolated pixelgenerator 1165 a to generate pixels by interpolating the pixels locatedon the path from the reference pixel to the subject pixel if necessary.

The multiplier 1166 multiplies the inter-pixel difference absolute valuetransformed by the function supplied from the function transformer 1164by the weight w_(s) supplied from the weight calculator 1165, and storesthe multiplication result in the storage portion 1167.

The SNDP calculator 1168 cumulatively adds the multiplication resultsstored in the storage portion 1167, and outputs the added value as theSNDP of the subject pixel.

The SNDP extraction processing performed by the SNDP extracting portion972 shown in FIG. 47 is described below with reference to the flowchartin FIG. 48.

In step S1081, the buffer 1161 temporarily stores HD images suppliedfrom the output phase converter 112.

In step S1082, the reference-pixel extracting portion 1162 selects anunprocessed pixel from the buffer 1161 as the subject pixel. In stepS1083, the reference-pixel extracting portion 1162 reads out referencepixels set for the subject pixel, and supplies the reference pixels tothe inter-pixel difference calculator 1163 and the weight calculator1165.

In step S1084, the inter-pixel difference calculator 1163 calculates theinter-pixel difference absolute value g between the subject pixel and anunprocessed reference pixel, and supplies the inter-pixel differenceabsolute value g to the function transformer 1164.

In step S1085, the function transformer 1164 transforms the inter-pixeldifference absolute value g supplied from the inter-pixel differencecalculator 1163 by a preset function F, and supplies the transformationresult to the multiplier 1166. More specifically, the functiontransformer 1164 transforms the inter-pixel difference absolute value gby, for example, the function F(g) indicated by the solid line in FIG.49, and outputs the transformation result to the multiplier 1166.According to the function F(g) shown in FIG. 49, the inter-pixeldifference absolute value g is transformed into a greater value if it issmaller than or equal to a predetermined value, and is transformed intoa smaller value if it is greater than the predetermined value.Accordingly, the inter-pixel difference absolute value g changes on theborder of the predetermined value.

In step S1086, the weight calculator 1165 determines whether it isnecessary to interpolate pixels between the subject pixel and thereference pixel. More specifically, if the subject pixel is P(0,0) andthe reference pixel is P(2,4) indicated by the double circle in FIG. 50,it is necessary to interpolate pixels P(0.5,1) and P(1.5,3) between thesubject pixel P(0,0) and the reference pixel P(2,4). The weightcalculator 1165 thus determines that it is necessary to interpolatepixels, and then proceeds to step S1087.

In step S1087, the weight calculator 1165 controls the interpolatedpixel generator 1165 a to generate pixels. For example, if the subjectpixel is P(0,0) and the reference pixel is P(2,4), as shown in FIG. 50,the weight calculator 1165 determines the values of the pixels P(0.5,1)and P(1.5,3) as the averages of (P(0,1)+P(1,1))/2 and (P(1,3)+P(1,1))/2,respectively. Another approach to interpolating pixels may be taken. Forexample, weights may be applied to the reference pixels and the weightedvalues may be added.

If the subject pixel is P(0,0) and the reference pixel is P(3,3)indicated by the cross in FIG. 50, pixels P(1,1) and P(2,2) exist on thepath between the subject pixel P(0,0) and the reference pixel P(3,3).Accordingly, the weight calculator 1165 determines in step S1086 that itis not necessary to interpolate pixels. The process then proceeds tostep S1088 by skipping step S1087.

In step S1088, the weight calculator 1165 controls the differencecalculator 1165 b to calculate the sum D of the inter-pixel differenceabsolute values on the path between the subject pixel and the referencepixel to determine the weight w_(g)=1−D/H (H is a constant), and thensupplies the weight w_(g) to the multiplier 1166. That is, in the caseof the subject pixel P(0,0) and the reference pixel P(2,4) in FIG. 50,the difference calculator 1165 b calculates the sum D|P(0,0)−P(0.5,1)|,|P(0.5,1)−P(1,2)|, |P(1,2)−P(1.5,3) |, and |P(1.5,3)−P(2,4)|. The weightcalculator 1165 also calculates the weight w_(g)=1−D/H and supplies theweight w_(g) to the multiplier 1166. The weight w_(g) may be set inassociation with a parameter, such as PNDP, for example, the weightw_(g)=1−D/PNDP or w_(g)=1−D/(√PNDP).

In step S1089, the multiplier 1166 multiplies the inter-pixel differenceabsolute value g supplied from the function transformer 1164 by theweight w_(g) supplied from the weight calculator 1165. In step S1090,the multiplier 1166 stores the multiplication result in the storageportion 1167.

In step S1091, the inter-pixel difference calculator 1163 checks for anunprocessed reference pixel. If there is an unprocessed pixel, theprocess returns to step S1084. That is, steps S1084 through S1091 arerepeated until inter-pixel difference absolute values between thesubject pixel and all the reference pixels are calculated.

If it is determined in step S1091 that there is no unprocessed referencepixel, the process proceeds to step S1092. In step S1092, theinter-pixel difference calculator 1163 supplies information that all thereference pixels have been processed to the SNDP calculator 1168. TheSNDP calculator 1168 determines the sum of the multiplication results inthe storage portion 1167, and outputs the sum as the SNDP.

In step S1093, the reference-pixel extracting portion 1162 determineswhether SNDPs have been calculated for all the pixels of the imagestored in the buffer 1161. If it is determined in step S1093 that SNDPshave not been calculated for all the pixels, the process returns to stepS1082. That is, steps S1082 through S1093 are repeated until SNDPs havebeen determined for all the pixels of the image stored in the buffer1161. If it is determined in step S1093 that SNDPs have been calculatedfor all the pixels, the SNDP extraction processing is completed.

According to the above-described processing, the SNDP representing thefeature of a flat portion in a narrow range of an image is determined.That is, the SNDP becomes greater if a narrow range containing thesubject pixel is more flat, and conversely, the SNDP becomes smaller ifthe narrow range containing the subject pixel is less flat, i.e., if itcontains many edges.

As stated above, the transform function F may be the function shown inFIG. 49. Alternatively, it may be associated with PNDP. For example, ifthe inter-pixel difference absolute value g is greater than or equal tothe threshold th11=PNDP/b11, the function F(g) may be expressed byF(g)=AA (AA is a constant). If the inter-pixel difference absolute valueg is smaller than or equal to th2=PNDP/b12, the function F(g) may beexpressed by F(g)=BB (BB is a constant). If the inter-pixel differenceabsolute value g is greater than the threshold th12=PNDP/b12 and smallerthan the threshold th11=BEP/b11, the function may be expressed byF(g)=(BB−AA)·(g−th11)/(th12−th11)+AA. Since it is sufficient if thethresholds th11 and th12 reflect the characteristic of the PNDP, thethresholds th1 and th12 may be modified as follows: th11=(PNDP)²/b11 andth12=(PNDP)²/b12, respectively, or th11=(√PNDP)/b11 andth12=(√PNDP)/b12, respectively.

As described above, as shown in FIG. 1, the natural-image predictionunit 131 and the artificial-image prediction unit 132 independentlyperform prediction processing, and then, the processing results arecombined based on the degree of artificiality Art generated by thenatural-image/artificial-image determining unit 114. Alternatively, anatural image and an artificial image may be separated from each otherin advance and may then be supplied to the natural-image prediction unit131 and the artificial-image prediction unit 132, respectively.

FIG. 51 illustrates the configuration of an image conversion device thatseparates a natural image and an artificial image in advance. In FIG.51, elements similar to those shown in FIG. 1 are designated with likereference numerals, and an explanation thereof is thus omitted.

The image conversion device 101 shown in FIG. 51 is different from thecounterpart shown in FIG. 1 in that a separator 1201 is disposed at thestage subsequent to the output phase converter 112 and an adder 1202 isdisposed at the stage after the natural-image prediction unit 131 andthe artificial-image prediction unit 132.

The separator 1201 separates an HD image supplied from the output phaseconverter 112 into a natural image and an artificial image in units ofpixels on the basis of the degree of artificiality Art supplied from thenatural-image/artificial-image determining unit 114, and then suppliesthe separated natural image and artificial image to the natural-imageprediction unit 131 and the artificial-image prediction unit 132,respectively.

The adder 1202 adds the pixels supplied from the natural-imageprediction unit 131 to the pixels supplied from the artificial-imageprediction unit 132 so that it relocates the high-quality pixels at theoriginal positions, thereby reconstructing an image.

The image conversion processing performed by the image conversion device101 shown in FIG. 51 is described below with reference to the flowchartin FIG. 52. Steps S1111, S1112, S1113, S1114, S1116, and S1117 in FIG.52 are similar to steps S1, S2, S3, S6, S4, and S5, respectively, inFIG. 2, and an explanation thereof is thus omitted.

In step S1115, the separator 1201 separates the HD image supplied fromthe output phase converter 112 into a natural image and an artificialimage in units of pixels on the basis of the degree of artificiality Artsupplied from the natural-image/artificial-image determining unit 114,and then supplies the separated natural image and artificial image tothe natural-image prediction unit 131 and the artificial-imageprediction unit 132, respectively. More specifically, when the degree ofartificiality is 1, the separator 1201 supplies the pixels to theartificial-image prediction unit 132. When the degree of artificialityis 0, the separator 1201 supplies the pixels to the natural-imageprediction unit 131. When the degree of artificiality is greater than 0and smaller than 1, the separator 1201 supplies, together withinformation concerning the degree of artificiality, the correspondingpixels to the natural-image prediction unit 131 and the artificial-imageprediction unit 132. In this case, the separator 1201 supplies, togetherwith the information concerning the separated pixels, the input image tothe natural-image prediction unit 131 and the artificial-imageprediction unit 132 for the sake of natural-image prediction processingand artificial-image prediction processing.

After the natural-image prediction processing and the artificial-imageprediction processing are performed in steps S1116 and S1117,respectively, in step S1118, the adder 1202 adds the pixels suppliedfrom the natural-image prediction unit 131 to the pixels supplied fromthe artificial-image prediction unit 132 to generate a high-qualityimage and outputs it. As stated above, when the degree of artificialityis greater than 0 and smaller than 1, the separator 1201 divides animage into natural-image pixels and artificial-image pixels on the basisof the degree of artificiality Art. In this case, the artificial-imagepixels are multiplied by the degree of artificiality Art and thenatural-image pixels are multiplied by the value obtained by subtractingthe degree of artificiality Art from 1 (1−Art), and then, the resultingpixels are added, thereby generating an image.

According to the above-described processing, advantages similar to thoseachieved by the image conversion device 101 shown in FIG. 1 can beobtained. In the image conversion device 101 shown in FIG. 51, pixels tobe supplied to the natural-image prediction unit 131 and theartificial-image prediction unit 132 are allocated in accordance withthe degree of artificiality Art. Accordingly, the processing loadimposed on the natural-image prediction unit 131 and theartificial-image prediction unit 132 can be reduced.

In the above-described example, the broad-range artificial imageboundary L1 is located above the broad-range natural image boundary L2,as shown in FIG. 32. This positional relationship, however, may bereversed depending on the features represented by the BEP and BFP.

More specifically, it is now assumed that the difference dynamic rangeof reference pixels, the pixel-value dynamic range using referencepixels, or the higher-level difference absolute value of the differenceabsolute values of the adjacent reference pixels rearranged in ascendingorder is used as the BEP and that the adjacent-pixel difference absolutevalues of the reference pixels greater than the threshold th aretransformed by the function and the sum of the resulting values for allthe reference pixels is used as the BFP. In this case, if the BEP andBFP are plotted on the horizontal axis and the vertical axis,respectively, the broad-range artificial image boundary is positionedabove the broad-range natural image boundary. Conversely, it is nowassumed that the difference dynamic range of reference pixels, thepixel-value dynamic range using reference pixels, or the higher-leveldifference absolute value of the difference absolute values of theadjacent reference pixels rearranged in ascending order is used as theBEP and that the higher-level difference absolute value of thedifference absolute values of the adjacent reference pixels rearrangedin ascending order is used as the BFP. In this case, if the BEP and BFPare plotted on the horizontal axis and the vertical axis, respectively,the broad-range artificial image boundary is positioned below thebroad-range natural image boundary.

The broad-range artificial image boundary L1 and the broad-range naturalimage boundary L2 may be changed by parameters used in performing imageconversion processing, such as the resolution, noise, and zoommagnification, so that the broad-range degree of artificiality Art_(b)can be increased or decreased.

The broad-range artificial image boundary L1 and the broad-range naturalimage boundary L2 shown in FIG. 53B are the same as those shown in FIG.32. In this case, if the resolution in image conversion processing isincreased, the broad-range artificial image boundary L1′ and thebroad-range natural image boundary L2′ are positioned, as shown in FIG.53A, lower than the broad-range artificial image boundary L1 and thebroad-range natural image boundary L2 shown in FIG. 53B. Accordingly,with the same broad-range feature, the broad-range degree ofartificiality Art_(b) becomes higher. On the other hand, if theresolution is decreased, the broad-range artificial image boundary L1″and the broad-range natural image boundary L2″ are positioned, as shownin FIG. 53C, higher than the broad-range artificial image boundary L1and the broad-range natural image boundary L2 shown in FIG. 53B.Accordingly, with the same broad-range feature, the broad-range degreeof artificiality Art_(b) becomes lower.

Similarly, the narrow-range natural image boundary L11 is located abovethe narrow-range artificial image boundary L12, as shown in FIG. 38.This positional relationship, however, may be reversed depending on thefeatures represented by the PNDP and SNDP.

It is now assumed that the pixel-value dynamic range of reference pixelsis used as the PNDP and that the sum of difference absolute values thatare weighted in accordance with the path from each reference pixel tothe subject pixel is used as the SNDP. In this case, if the SNDP andPNDP are plotted on the horizontal axis and the vertical axis,respectively, the narrow-range artificial image boundary L12 ispositioned above the narrow-range natural image boundary L11. It is nowassumed that the pixel-value dynamic range of a long tap is used as thePNDP and that the pixel-value dynamic range of short taps is used as theSNDP. In this case, if the SNDP and PNDP are plotted on the horizontalaxis and the vertical axis, respectively, the narrow-range artificialimage boundary L12 is positioned below the narrow-range natural imageboundary L11.

The narrow-range natural image boundary L11 and the narrow-rangeartificial image boundary L12 may be changed by parameters used inperforming image conversion processing, such as the resolution, noise,and zoom magnification, so that the narrow-range degree of artificialityArt_(n) can be increased or decreased.

The narrow-range natural image boundary L11 and the narrow-rangeartificial image boundary L12 shown in FIG. 54B are the same as thoseshown in FIG. 38. In this case, if the noise in image conversionprocessing is increased, the narrow-range natural image boundary L11′and the narrow-range artificial image boundary L12′ are positioned, asshown in FIG. 54A, lower than the narrow-range natural image boundaryL11 and the narrow-range artificial image boundary L12 shown in FIG.54B. Accordingly, with the same narrow-range feature, the narrow-rangedegree of artificiality Art_(n) becomes higher. On the other hand, ifthe noise is decreased, the narrow-range natural image boundary L11″ andthe narrow-range artificial image boundary L12″ are positioned, as shownin FIG. 54C, higher than the narrow-range natural image boundary L11 andthe narrow-range artificial image boundary L12 shown in FIG. 54B.Accordingly, with the same narrow-range feature, the narrow-range degreeof artificiality Art_(n) becomes lower.

In the above-described example, the BEP and BFP are used as thebroad-range features, while the PNDP and SNDP are used as thenarrow-range features. However, parameters other than the BEP and BFPmay be used as long as they can represent edge or flat portions in abroad range. The same applies to the PNDP and SNDP. Parameters otherthan the PNDP and SNDP may be used as long as they can represent thinlines, edges, points, flat portions in the vicinity of edges, orgradation in a narrow range. Additionally, in the above-describedexample, two types of features are used for representing each of thebroad-range feature and narrow-range feature. Alternatively, more thantwo types of features may be used, in which case, instead of thetwo-dimensional artificial-image and natural-image boundaries shown inFIG. 32 or 38, the n-dimensional boundary lines or boundary planes basedon the number n of features may be employed. However, basic operationsin the n-dimensional space, such as determining whether a subject pixelbelongs to a natural image or an artificial image or determining theratio of the distance from the subject pixel to an artificial imageboundary line or plane to that to a natural image boundary line orplane, are similar to the case where two types of features are employed.

As described above, the quality of an overall image can be enhanced bydistinguishing, in units of pixels, natural image components andartificial image components contained in the image from each other andby performing optimal processing on each of the components.

The above-described series of processing operations may be executed byhardware or software. If software is used, a corresponding softwareprogram is installed from a recording medium into a computer built indedicated hardware or a computer, such as a general-purpose computer,that can execute various functions by installing various programstherein.

FIG. 55 is a block diagram illustrating the configuration of a personalcomputer when the electrical internal configuration of the imageconversion device 101 shown in FIG. 1 or 51 is implemented by software.A central processing unit (CPU) 2001 controls the overall operation ofthe personal computer. The CPU 2001 also executes a program stored in aread only memory (ROM) 2002 in response to an instruction input from aninput unit 2006, such as a keyboard and a mouse, by a user via a bus2004 and an input/output interface 2005. The CPU 2001 also loads, into arandom access memory (RAM) 2003, a program read from a removable disk2011, such as a magnetic disk, an optical disc, a magneto-optical (MO)disk, or a semiconductor memory, connected to a drive 2010, andinstalled into a storage unit 2008, and then executes the loadedprogram. With this configuration, the functions of the image conversiondevice 101 shown in FIG. 1 or 51 can be implemented by software. The CPU2001 also controls a communication unit 2009 to communicate with anexternal source and sends and receives data.

The recording medium on which the program is recorded may be formed of apackage medium, as shown in FIG. 55, which is distributed for providingthe program to users separately from the computer, such as the removablemedium 2011, including a magnetic disk (including flexible disk), anoptical disc (including a compact disc read only memory (CD-ROM) and adigital versatile disk (DVD)), a MO disk (including a mini-disc (MD)),and a semiconductor memory. Alternatively, the recording medium may beformed of the ROM 2002 or a hard disk contained in the recording unit2008, on which the program is recorded, which is provided to users whilebeing built in the computer.

In this specification, steps forming the program recorded on therecording medium may be executed in chronological order described in thespecification. Alternatively, they may be executed in parallel orindividually.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

1. An image processing apparatus comprising: broad-range featureextraction means for extracting broad-range features including aplurality of types of broad-range features from pixels located in apredetermined area in relation to a subject pixel of a first image;broad-range degree-of-artificiality calculation means for calculating,in a multidimensional space represented by the plurality of types ofbroad-range features included in the broad-range features extracted bythe broad-range feature extraction means, from a positional relationshipof the plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image; narrow-range featureextraction means for extracting narrow-range features including aplurality of types of narrow-range features from pixels located in thepredetermined area in relation to the subject pixel of the first image;narrow-range degree-of-artificiality calculation means for calculating,in a multidimensional space represented by the plurality of types ofnarrow-range features included in the narrow-range features extracted bythe narrow-range feature extraction means, from a positionalrelationship of the plurality of types of narrow-range featuresaccording to a statistical distribution range of the artificial image ofthe first image, a narrow-range degree of artificiality representing adegree by which the plurality of types of narrow-range features belongto the statistical distribution range of the artificial image; anddegree-of-artificiality calculation means for calculating a degree ofartificiality of the subject pixel by combining the broad-range degreeof artificiality and the narrow-range degree of artificiality.
 2. Theimage processing apparatus according to claim 1, further comprising:first prediction means for predicting, from the first image, a secondimage which is obtained by increasing the quality of the artificialimage; second prediction means for predicting, from the first image, athird image which is obtained by increasing the quality of a naturalimage exhibiting a large number of grayscale levels and indistinctedges; and synthesizing means for combining the second image and thethird image on the basis of the degree of artificiality.
 3. The imageprocessing apparatus according to claim 2, wherein the first predictionmeans includes first classification means for classifying pixels of thesecond image into first classes, first storage means for storing a firstprediction coefficient for each of the first classes, the firstprediction coefficient being obtained by conducting learning by using aplurality of artificial images, and first computation means fordetermining, from the first image, the second image having a higherquality than the first image by performing computation using the firstimage and the first prediction coefficients for the first classes intowhich the pixels of the second image are classified, and wherein thesecond prediction means includes second classification means forclassifying pixels of the third image into second classes, secondstorage means for storing a second prediction coefficient for each ofthe second classes, the second prediction coefficient being obtained byconducting learning by using a plurality of natural images, and secondcomputation means for determining the third image from the first imageby performing computation using the first image and the secondprediction coefficients for the second classes into which the pixels ofthe third image are classified.
 4. The image processing apparatusaccording to claim 1, wherein the broad-range degree-of-artificialitycalculation means includes broad-range artificial-image distributionrange storage means for storing the statistical distribution range ofthe artificial image of the first image in the multidimensional spacerepresented by the plurality of types of broad-range features, and thebroad-range degree-of-artificiality calculation means calculates thebroad-range degree of artificiality from the positional relationship ofthe plurality of types of broad-range features extracted by thebroad-range feature extraction means according to the statisticaldistribution range of the artificial image in the multidimensional spacestored in the broad-range artificial-image distribution range storagemeans.
 5. The image processing apparatus according to claim 1, whereinthe narrow-range degree-of-artificiality calculation means includesnarrow-range artificial-image distribution range storage means forstoring the statistical distribution range of the artificial image ofthe first image in the multidimensional space represented by theplurality of types of narrow-range features, and the narrow-rangedegree-of-artificiality calculation means calculates the narrow-rangedegree of artificiality from the positional relationship of theplurality of types of narrow-range features extracted by thenarrow-range feature extraction means according to the statisticaldistribution range of the artificial image in the multidimensional spacestored in the narrow-range artificial-image distribution range storagemeans.
 6. The image processing apparatus according to claim 1, whereinthe broad-range feature extraction means includes edge featureextraction means for extracting a feature representing the presence ofan edge from the pixels located in the predetermined area, and flatfeature extraction means for extracting a feature representing thepresence of a flat portion from the pixels located in the predeterminedarea.
 7. The image processing apparatus according to claim 6, whereinthe edge feature extraction means extracts, as the feature representingthe presence of an edge, a difference dynamic range of the pixelslocated in the predetermined area by using a difference of pixel valuesbetween the subject pixel and each of the pixels located in thepredetermined area.
 8. The image processing apparatus according to claim7, wherein the edge feature extraction means extracts, as the featurerepresenting the presence of an edge, a difference dynamic range of thepixels located in the predetermined area by using a difference of thepixel values between the subject pixel and each of the pixels located inthe predetermined area after applying a weight to the difference of thepixel values in accordance with a distance therebetween.
 9. The imageprocessing apparatus according to claim 6, wherein the edge featureextraction means extracts, as the feature representing the presence ofan edge, a predetermined order of higher levels of difference absolutevalues between adjacent pixels of the pixels located in thepredetermined area.
 10. The image processing apparatus according toclaim 6, wherein the edge feature extraction means extracts, as thefeature representing the presence of an edge, the average of higherfirst through second levels of difference absolute values betweenadjacent pixels of the pixels located in the predetermined area or thesum of the higher first through second levels of the difference absolutevalues after applying a weight to each of the difference absolutevalues.
 11. The image processing apparatus according to claim 6, whereinthe flat feature extraction means extracts, as the feature representingthe presence of a flat portion, the number of difference absolute valuesbetween adjacent pixels of the pixels located in the predetermined areawhich are smaller than or equal to a predetermined threshold.
 12. Theimage processing apparatus according to claim 11, wherein thepredetermined threshold is set based on the feature representing thepresence of an edge.
 13. The image processing apparatus according toclaim 6, wherein the flat feature extraction means extracts, as thefeature representing the presence of a flat portion, the sum ofdifference absolute values between adjacent pixels of the pixels locatedin the predetermined area after transforming the difference absolutevalues by a predetermined function.
 14. The image processing apparatusaccording to claim 13, wherein the flat feature extraction meansextracts, as the feature representing the presence of a flat portion,the sum of difference absolute values between the adjacent pixels of thepixels located in the predetermined area after transforming thedifference absolute values by a predetermined function and afterapplying a weight to each of the transformed difference absolute valuesin accordance with a distance from the subject pixel to each of thepixels located in the predetermined area.
 15. The image processingapparatus according to claim 13, wherein the predetermined function is afunction associated with the feature representing the presence of anedge.
 16. The image processing apparatus according to claim 6, whereinthe flat feature extraction means extracts, as the feature representingthe presence of a flat portion, a predetermined order of lower levels ofdifference absolute values between adjacent pixels of the pixels locatedin the predetermined area.
 17. The image processing apparatus accordingto claim 6, wherein the flat feature extraction means extracts, as thefeature representing the presence of a flat portion, the average oflower first through second levels of difference absolute values betweenadjacent pixels of the pixels located in the predetermined area or thesum of the lower first through second levels of the difference absolutevalues after applying a weight to each of the difference absolutevalues.
 18. The image processing apparatus according to claim 1, whereinthe narrow-range feature extraction means extracts, from the pixelslocated in the predetermined area, the narrow-range features includingtwo types of features selected from features representing thin lines,edges, points, flat portions in the vicinity of edges, and gradation.19. The image processing apparatus according to claim 18, wherein thenarrow-range feature extraction means includes first narrow-rangefeature extraction means for extracting, as a first feature of thenarrow-range features, a pixel-value dynamic range obtained bysubtracting a minimum pixel value from a maximum pixel value of pixelslocated in a first area included in the predetermined area, and secondnarrow-range feature extraction means for extracting, as a secondfeature of the narrow-range features, a pixel-value dynamic rangeobtained by subtracting a minimum pixel value from a maximum pixel valueof pixels located in a second area including the subject pixel andincluded in the first area.
 20. The image processing apparatus accordingto claim 19, wherein the second narrow-range feature extraction meansextracts, as the second feature, a minimum pixel-value dynamic range ofpixel-value dynamic ranges obtained from a plurality of the secondareas.
 21. The image processing apparatus according to claim 19, whereinthe first narrow-range feature extraction means extracts, as the firstfeature of the narrow-range features, a pixel-value dynamic range of thepixel values of the pixels located in the predetermined area, andwherein the second narrow-range feature extraction means extracts, asthe second feature of the narrow-range features, the sum of differenceabsolute values between the subject pixel and the pixels located in thepredetermined area after transforming the difference absolute values bya predetermined function and after applying a weight to each of thetransformed difference absolute values.
 22. The image processingapparatus according to claim 21, wherein the second narrow-range featureextraction means includes weight calculation means for calculating theweight in accordance with the sum of the difference absolute valuesbetween adjacent pixels of all the pixels located on a path from thesubject pixel to the pixels located in the first area.
 23. The imageprocessing apparatus according to claim 21, wherein the predeterminedfunction is a function associated with the first feature.
 24. An imageprocessing method comprising the steps of: extracting broad-rangefeatures including a plurality of types of broad-range features frompixels located in a predetermined area in relation to a subject pixel ofa first image; calculating with a processor, in a multidimensional spacerepresented by the plurality of types of broad-range features includedin the extracted broad-range features, from a positional relationship ofthe plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image; extracting narrow-rangefeatures including a plurality of types of narrow-range features frompixels located in the predetermined area in relation to the subjectpixel of the first image; calculating, in a multidimensional spacerepresented by the plurality of types of narrow-range features includedin the extracted narrow-range features, from a positional relationshipof the plurality of types of narrow-range features according to astatistical distribution range of the artificial image of the firstimage, a narrow-range degree of artificiality representing a degree bywhich the plurality of types of narrow-range features belong to thestatistical distribution range of the artificial image; and calculatinga degree of artificiality of the subject pixel by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality.
 25. A non-transitory computer readable medium havingrecorded thereon a computer-readable program that when executed by aprocessor, performs steps comprising: extracting broad-range featuresincluding a plurality of types of broad-range features from pixelslocated in a predetermined area in relation to a subject pixel of afirst image; calculating, in a multidimensional space represented by theplurality of types of broad-range features included in the extractedbroad-range features, from a positional relationship of the plurality oftypes of broad-range features according to a statistical distributionrange of an artificial image, which exhibits a small number of grayscalelevels and distinct edges, of the first image, a broad-range degree ofartificiality representing a degree by which the plurality of types ofbroad-range features belong to the statistical distribution range of theartificial image; extracting narrow-range features including a pluralityof types of narrow-range features from pixels located in thepredetermined area in relation to the subject pixel of the first image;calculating, in a multidimensional space represented by the plurality oftypes of narrow-range features included in the extracted narrow-rangefeatures, from a positional relationship of the plurality of types ofnarrow-range features according to a statistical distribution range ofthe artificial image of the first image, a narrow-range degree ofartificiality representing a degree by which the plurality of types ofnarrow-range features belong to the statistical distribution range ofthe artificial image; and calculating a degree of artificiality of thesubject pixel by combining the broad-range degree of artificiality andthe narrow-range degree of artificiality.
 26. An image processingapparatus comprising: a broad-range feature extraction unit configuredto extract broad-range features including a plurality of types ofbroad-range features from pixels located in a predetermined area inrelation to a subject pixel of a first image; a broad-rangedegree-of-artificiality calculator configured to calculate, in amultidimensional space represented by the plurality of types ofbroad-range features included in the broad-range features extracted bythe broad-range feature extraction unit, from a positional relationshipof the plurality of types of broad-range features according to astatistical distribution range of an artificial image, which exhibits asmall number of grayscale levels and distinct edges, of the first image,a broad-range degree of artificiality representing a degree by which theplurality of types of broad-range features belong to the statisticaldistribution range of the artificial image; a narrow-range featureextraction unit configured to extract narrow-range features including aplurality of types of narrow-range features from pixels located in thepredetermined area in relation to the subject pixel of the first image;a narrow-range degree-of-artificiality calculator having a processorconfigured to calculate, in a multidimensional space represented by theplurality of types of narrow-range features included in the narrow-rangefeatures extracted by the narrow-range feature extraction unit, from apositional relationship of the plurality of types of narrow-rangefeatures according to a statistical distribution range of the artificialimage of the first image, a narrow-range degree of artificialityrepresenting a degree by which the plurality of types of narrow-rangefeatures belong to the statistical distribution range of the artificialimage; and a degree-of-artificiality calculator configured to calculatea degree of artificiality of the subject pixel by combining thebroad-range degree of artificiality and the narrow-range degree ofartificiality.